Jun 13 00:53:48.382338 ip-10-0-137-107 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 13 00:53:48.382351 ip-10-0-137-107 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 13 00:53:48.382359 ip-10-0-137-107 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 13 00:53:48.382599 ip-10-0-137-107 systemd[1]: Failed to start Kubernetes Kubelet. Jun 13 00:53:57.823896 ip-10-0-137-107 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jun 13 00:53:57.823911 ip-10-0-137-107 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 85002323efba4fc3a56363ca8ec04000 -- Jun 13 00:56:00.312052 ip-10-0-137-107 systemd[1]: Starting Kubernetes Kubelet... Jun 13 00:56:00.698640 ip-10-0-137-107 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 13 00:56:00.698640 ip-10-0-137-107 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 13 00:56:00.698640 ip-10-0-137-107 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 13 00:56:00.698640 ip-10-0-137-107 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 13 00:56:00.698640 ip-10-0-137-107 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 13 00:56:00.699148 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.698699 2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 13 00:56:00.703910 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703896 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 13 00:56:00.703910 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703909 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703913 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703916 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703919 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703922 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703925 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703927 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703930 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703932 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703935 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703938 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703941 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703944 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703947 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703950 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703952 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703955 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703958 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703962 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 13 00:56:00.703969 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703965 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703968 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703972 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703974 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703977 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703980 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703983 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703986 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703988 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703991 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703993 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703996 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.703998 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704001 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704003 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704007 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704010 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704013 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704016 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704018 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 13 00:56:00.704511 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704021 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704023 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704026 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704029 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704031 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704034 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704037 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704040 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704043 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704046 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704049 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704052 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704054 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704057 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704060 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704062 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704065 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704067 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704070 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704072 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 13 00:56:00.705211 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704075 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704077 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704079 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704082 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704084 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704087 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704089 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704091 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704094 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704096 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704098 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704101 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704104 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704109 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704111 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704114 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704116 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704119 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704122 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704125 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 13 00:56:00.705682 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704128 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704130 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704132 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704135 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704138 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704140 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704143 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704146 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704148 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704151 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704154 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704156 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704159 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704161 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704164 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704166 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704169 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704171 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.704173 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705959 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 13 00:56:00.706154 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705970 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705973 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705977 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705980 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705982 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705985 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705988 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705990 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705993 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705995 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.705998 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706000 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706003 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706005 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706008 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706010 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706013 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706015 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706017 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706027 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 13 00:56:00.706634 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706030 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706032 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706035 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706037 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706040 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706042 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706045 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706048 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706050 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706053 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706055 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706058 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706061 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706064 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706070 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706073 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706075 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706078 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706080 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706083 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 13 00:56:00.707132 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706085 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706088 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706090 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706093 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706095 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706097 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706100 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706102 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706104 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706108 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706110 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706114 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706117 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706120 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706122 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706125 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706127 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706129 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706132 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 13 00:56:00.707632 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706136 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706138 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706141 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706143 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706146 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706149 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706152 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706155 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706158 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706161 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706163 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706167 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706171 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706174 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706177 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706180 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706183 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706186 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706189 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 13 00:56:00.708093 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706192 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706194 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706197 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706200 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706203 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706205 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706207 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706210 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706213 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706215 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706218 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706220 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706223 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706225 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706228 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706230 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706232 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706235 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706237 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 13 00:56:00.708544 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706241 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706603 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706609 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706613 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706615 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706618 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706621 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706623 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706626 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706628 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706631 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706634 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706636 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706639 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706641 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706644 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706647 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706650 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706653 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706655 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 13 00:56:00.708998 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706657 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706660 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706664 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706667 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706671 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706674 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706676 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706679 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706682 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706684 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706687 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706690 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706692 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706695 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706697 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706700 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706702 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706706 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706709 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 13 00:56:00.709469 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706712 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706714 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706717 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706720 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706722 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706725 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706727 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706730 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706732 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706735 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706737 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706740 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706742 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706745 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706747 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706749 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706752 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706754 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706757 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706759 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 13 00:56:00.709924 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706761 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706764 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706766 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706768 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706771 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706773 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706791 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706795 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706799 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706802 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706805 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706807 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706810 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706812 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706815 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706818 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706820 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706823 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706825 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706828 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 13 00:56:00.710401 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706830 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706832 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706835 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706837 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706842 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706846 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706849 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706852 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706854 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706857 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706859 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706861 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706864 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706866 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706868 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706871 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706873 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706876 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706878 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 13 00:56:00.710880 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706883 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.706886 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706949 2552 flags.go:64] FLAG: --address="0.0.0.0" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706955 2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706961 2552 flags.go:64] FLAG: --anonymous-auth="true" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706965 2552 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706969 2552 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706972 2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706976 2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706980 2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706983 2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706986 2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706989 2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706992 2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706995 2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.706998 2552 flags.go:64] FLAG: --cgroup-root="" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707001 2552 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707004 2552 flags.go:64] FLAG: --client-ca-file="" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707007 2552 flags.go:64] FLAG: --cloud-provider="external" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707010 2552 flags.go:64] FLAG: --cluster-dns="[]" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707013 2552 flags.go:64] FLAG: --cluster-domain="" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707016 2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707019 2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707022 2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 13 00:56:00.711336 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707025 2552 flags.go:64] FLAG: --container-log-max-files="5" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707029 2552 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707032 2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707035 2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707038 2552 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707041 2552 flags.go:64] FLAG: --contention-profiling="false" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707044 2552 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707047 2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707055 2552 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707058 2552 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707062 2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707065 2552 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707068 2552 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707071 2552 flags.go:64] FLAG: --enable-load-reader="false" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707074 2552 flags.go:64] FLAG: --enable-server="true" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707077 2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707081 2552 flags.go:64] FLAG: --event-burst="100" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707084 2552 flags.go:64] FLAG: --event-qps="50" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707087 2552 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707090 2552 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707093 2552 flags.go:64] FLAG: --eviction-hard="" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707096 2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707099 2552 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707102 2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707105 2552 flags.go:64] FLAG: --eviction-soft="" Jun 13 00:56:00.711909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707107 2552 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707110 2552 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707114 2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707116 2552 flags.go:64] FLAG: --experimental-mounter-path="" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707119 2552 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707122 2552 flags.go:64] FLAG: --fail-swap-on="true" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707125 2552 flags.go:64] FLAG: --feature-gates="" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707128 2552 flags.go:64] FLAG: --file-check-frequency="20s" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707131 2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707134 2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707137 2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707140 2552 flags.go:64] FLAG: --healthz-port="10248" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707143 2552 flags.go:64] FLAG: --help="false" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707146 2552 flags.go:64] FLAG: --hostname-override="ip-10-0-137-107.ec2.internal" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707149 2552 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707153 2552 flags.go:64] FLAG: --http-check-frequency="20s" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707155 2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707159 2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707162 2552 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707164 2552 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707167 2552 flags.go:64] FLAG: --image-service-endpoint="" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707169 2552 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707172 2552 flags.go:64] FLAG: --kube-api-burst="100" Jun 13 00:56:00.712484 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707175 2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707178 2552 flags.go:64] FLAG: --kube-api-qps="50" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707181 2552 flags.go:64] FLAG: --kube-reserved="" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707184 2552 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707186 2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707189 2552 flags.go:64] FLAG: --kubelet-cgroups="" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707192 2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707194 2552 flags.go:64] FLAG: --lock-file="" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707197 2552 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707200 2552 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707202 2552 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707208 2552 flags.go:64] FLAG: --log-json-split-stream="false" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707211 2552 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707214 2552 flags.go:64] FLAG: --log-text-split-stream="false" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707217 2552 flags.go:64] FLAG: --logging-format="text" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707219 2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707223 2552 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707225 2552 flags.go:64] FLAG: --manifest-url="" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707228 2552 flags.go:64] FLAG: --manifest-url-header="" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707232 2552 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707235 2552 flags.go:64] FLAG: --max-open-files="1000000" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707239 2552 flags.go:64] FLAG: --max-pods="110" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707241 2552 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707244 2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707248 2552 flags.go:64] FLAG: --memory-manager-policy="None" Jun 13 00:56:00.713102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707251 2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707254 2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707257 2552 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707260 2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707266 2552 flags.go:64] FLAG: --node-status-max-images="50" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707269 2552 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707272 2552 flags.go:64] FLAG: --oom-score-adj="-999" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707275 2552 flags.go:64] FLAG: --pod-cidr="" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707278 2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707283 2552 flags.go:64] FLAG: --pod-manifest-path="" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707286 2552 flags.go:64] FLAG: --pod-max-pids="-1" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707289 2552 flags.go:64] FLAG: --pods-per-core="0" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707291 2552 flags.go:64] FLAG: --port="10250" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707294 2552 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707297 2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-074d08f0a84f45538" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707301 2552 flags.go:64] FLAG: --qos-reserved="" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707303 2552 flags.go:64] FLAG: --read-only-port="10255" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707306 2552 flags.go:64] FLAG: --register-node="true" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707309 2552 flags.go:64] FLAG: --register-with-taints="" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707315 2552 flags.go:64] FLAG: --registry-burst="10" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707319 2552 flags.go:64] FLAG: --registry-qps="5" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707321 2552 flags.go:64] FLAG: --reserved-cpus="" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707324 2552 flags.go:64] FLAG: --reserved-memory="" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707327 2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707330 2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 13 00:56:00.713692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707333 2552 flags.go:64] FLAG: --rotate-certificates="false" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707336 2552 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707338 2552 flags.go:64] FLAG: --runonce="false" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707341 2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707344 2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707347 2552 flags.go:64] FLAG: --seccomp-default="false" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707352 2552 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707355 2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707357 2552 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707360 2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707363 2552 flags.go:64] FLAG: --storage-driver-password="root" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707366 2552 flags.go:64] FLAG: --storage-driver-secure="false" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707369 2552 flags.go:64] FLAG: --storage-driver-table="stats" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707371 2552 flags.go:64] FLAG: --storage-driver-user="root" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707374 2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707377 2552 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707380 2552 flags.go:64] FLAG: --system-cgroups="" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707382 2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707387 2552 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707390 2552 flags.go:64] FLAG: --tls-cert-file="" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707393 2552 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707396 2552 flags.go:64] FLAG: --tls-min-version="" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707399 2552 flags.go:64] FLAG: --tls-private-key-file="" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707401 2552 flags.go:64] FLAG: --topology-manager-policy="none" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707404 2552 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 13 00:56:00.714375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707407 2552 flags.go:64] FLAG: --topology-manager-scope="container" Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707412 2552 flags.go:64] FLAG: --v="2" Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707415 2552 flags.go:64] FLAG: --version="false" Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707420 2552 flags.go:64] FLAG: --vmodule="" Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707428 2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.707431 2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707531 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707534 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707537 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707540 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707543 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707546 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707549 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707554 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707557 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707559 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707563 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707567 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707569 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707572 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 13 00:56:00.714971 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707575 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707578 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707580 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707583 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707586 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707588 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707591 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707594 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707596 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707599 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707601 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707604 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707606 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707610 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707612 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707615 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707617 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707620 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707622 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707625 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 13 00:56:00.715455 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707628 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707630 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707633 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707636 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707638 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707642 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707644 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707647 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707649 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707652 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707654 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707657 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707659 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707661 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707664 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707666 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707669 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707671 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707674 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707676 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 13 00:56:00.715948 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707679 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707681 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707684 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707687 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707689 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707692 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707695 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707697 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707700 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707703 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707706 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707709 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707711 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707715 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707719 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707723 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707726 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707730 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707733 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 13 00:56:00.716444 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707735 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707738 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707741 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707743 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707746 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707748 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707751 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707753 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707756 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707758 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707760 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707763 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707766 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707768 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707770 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707773 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707775 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707790 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707794 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707797 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 13 00:56:00.716913 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707799 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707802 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707804 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707808 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707810 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.707813 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.708428 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.714403 2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.714417 2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.715307 2552 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.715315 2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715366 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715372 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715375 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715378 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 13 00:56:00.717383 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715381 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715384 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715387 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715390 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715393 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715395 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715398 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715401 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715404 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715407 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715409 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715412 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715414 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715417 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715419 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715422 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715424 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715426 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715429 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 13 00:56:00.717745 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715431 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715433 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715436 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715438 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715441 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715443 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715447 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715450 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715452 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715455 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715457 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715461 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715463 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715466 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715468 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715471 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715473 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715476 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715478 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715480 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 13 00:56:00.718192 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715483 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715485 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715488 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715491 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715493 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715496 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715498 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715501 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715503 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715506 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715508 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715510 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715513 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715515 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715518 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715520 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715523 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715525 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715528 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715532 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 13 00:56:00.718671 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715535 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715538 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715541 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715544 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715548 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715551 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715554 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715556 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715559 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715562 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715564 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715567 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715570 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715573 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715577 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715581 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715584 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715586 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715589 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 13 00:56:00.719149 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715592 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715595 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715597 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715600 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715602 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715605 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715607 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715610 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715612 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715615 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715618 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715620 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715623 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715625 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715628 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715630 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 13 00:56:00.719582 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715633 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.715637 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715736 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715740 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715743 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715745 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715747 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715750 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715753 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715756 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715758 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715761 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715764 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715766 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715769 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 13 00:56:00.720090 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715771 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715773 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715789 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715793 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715796 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715799 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715801 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715803 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715806 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715809 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715811 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715813 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715816 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715818 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715821 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715824 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715826 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715830 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715834 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 13 00:56:00.720449 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715837 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715840 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715842 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715844 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715847 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715849 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715851 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715854 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715856 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715859 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715861 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715864 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715867 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715869 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715871 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715874 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715876 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715878 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715881 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715883 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 13 00:56:00.720900 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715885 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715888 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715890 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715893 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715895 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715897 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715900 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715903 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715905 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715908 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715910 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715913 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715915 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715917 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715920 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715922 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715925 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715927 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715929 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715932 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 13 00:56:00.721347 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715934 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715936 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715939 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715941 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715944 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715946 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715950 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715954 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715957 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715961 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715963 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715966 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715968 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715971 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715973 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715976 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715978 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715981 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715983 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 13 00:56:00.721815 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715986 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715988 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715990 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715993 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715995 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.715998 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.716000 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:00.716003 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.716007 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.716099 2552 server.go:962] "Client rotation is on, will bootstrap in background" Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.717996 2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.719444 2552 server.go:1019] "Starting client certificate rotation" Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.719539 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 13 00:56:00.722260 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.720123 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 13 00:56:00.722714 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.722700 2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 13 00:56:00.724084 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.723689 2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 13 00:56:00.736054 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.736035 2552 log.go:25] "Validated CRI v1 runtime API" Jun 13 00:56:00.738863 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.738848 2552 log.go:25] "Validated CRI v1 image API" Jun 13 00:56:00.739647 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.739635 2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 13 00:56:00.741738 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.741722 2552 fs.go:135] Filesystem UUIDs: map[4f64323f-e775-4d67-8a61-944f9561c1e6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e02c6ead-421b-4302-b144-b613dfd1d0dc:/dev/nvme0n1p3] Jun 13 00:56:00.741809 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.741738 2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 13 00:56:00.746446 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.746350 2552 manager.go:216] Machine: {Timestamp:2026-06-13 00:56:00.745071889 +0000 UTC m=+0.319000770 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200927 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d175ba056b1991e0a20e5e939c79e SystemUUID:ec2d175b-a056-b199-1e0a-20e5e939c79e BootID:85002323-efba-4fc3-a563-63ca8ec04000 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:21:05:61:0a:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:21:05:61:0a:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:b6:1a:be:dd:24 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 13 00:56:00.746446 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.746444 2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 13 00:56:00.746549 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.746512 2552 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 13 00:56:00.749503 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.749479 2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 13 00:56:00.749621 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.749506 2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-107.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 13 00:56:00.750192 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.750183 2552 topology_manager.go:138] "Creating topology manager with none policy" Jun 13 00:56:00.750227 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.750195 2552 container_manager_linux.go:309] "Creating device plugin manager" Jun 13 00:56:00.750227 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.750202 2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 13 00:56:00.750961 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.750946 2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 13 00:56:00.752181 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.752171 2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 13 00:56:00.755898 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.755887 2552 state_mem.go:36] "Initialized new in-memory state store" Jun 13 00:56:00.756008 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.755999 2552 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 13 00:56:00.756048 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.756028 2552 kubelet.go:486] "Attempting to sync node with API server" Jun 13 00:56:00.756048 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.756044 2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 13 00:56:00.756100 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.756057 2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 13 00:56:00.756100 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.756065 2552 kubelet.go:398] "Adding apiserver pod source" Jun 13 00:56:00.756100 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.756073 2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 13 00:56:00.756572 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.756551 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 13 00:56:00.756627 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.756381 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 13 00:56:00.756667 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.756652 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 13 00:56:00.758660 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.758645 2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 13 00:56:00.759990 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.759969 2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 13 00:56:00.760074 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.759999 2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 13 00:56:00.760141 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760131 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 13 00:56:00.760175 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760146 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 13 00:56:00.760175 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760151 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 13 00:56:00.760175 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760156 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 13 00:56:00.760175 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760164 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 13 00:56:00.760175 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760169 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 13 00:56:00.760737 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760729 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 13 00:56:00.760765 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760743 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 13 00:56:00.760765 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760750 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 13 00:56:00.760765 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760755 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 13 00:56:00.760864 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760767 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 13 00:56:00.760864 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.760775 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 13 00:56:00.761422 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.761413 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 13 00:56:00.761422 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.761421 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 13 00:56:00.764199 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.764187 2552 server.go:1268] "Started kubelet" Jun 13 00:56:00.764321 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.764281 2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 13 00:56:00.764373 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.764341 2552 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 13 00:56:00.764420 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.764295 2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 13 00:56:00.764903 ip-10-0-137-107 systemd[1]: Started Kubernetes Kubelet. Jun 13 00:56:00.769308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.769279 2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 13 00:56:00.771304 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.771288 2552 server.go:310] "Adding debug handlers to kubelet server" Jun 13 00:56:00.773714 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.773695 2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-107.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 13 00:56:00.773714 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.773707 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-107.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 13 00:56:00.773847 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.773758 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 13 00:56:00.774582 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.773625 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-107.ec2.internal.18b87df8f06e71ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-107.ec2.internal,UID:ip-10-0-137-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-107.ec2.internal,},FirstTimestamp:2026-06-13 00:56:00.764162541 +0000 UTC m=+0.338091418,LastTimestamp:2026-06-13 00:56:00.764162541 +0000 UTC m=+0.338091418,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-107.ec2.internal,}" Jun 13 00:56:00.775714 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.775678 2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 13 00:56:00.777064 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.777047 2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 13 00:56:00.777064 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.777056 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 13 00:56:00.777189 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.777133 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:00.777692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.777657 2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 13 00:56:00.777692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.777665 2552 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 13 00:56:00.777692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.777677 2552 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 13 00:56:00.777882 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.777792 2552 reconstruct.go:97] "Volume reconstruction finished" Jun 13 00:56:00.777882 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.777800 2552 reconciler.go:29] "Reconciler: start to sync state" Jun 13 00:56:00.777882 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.777799 2552 factory.go:55] Registering systemd factory Jun 13 00:56:00.777882 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.777823 2552 factory.go:223] Registration of the systemd container factory successfully Jun 13 00:56:00.778069 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.778027 2552 factory.go:153] Registering CRI-O factory Jun 13 00:56:00.778069 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.778038 2552 factory.go:223] Registration of the crio container factory successfully Jun 13 00:56:00.778158 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.778078 2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 13 00:56:00.778158 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.778094 2552 factory.go:103] Registering Raw factory Jun 13 00:56:00.778158 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.778103 2552 manager.go:1196] Started watching for new ooms in manager Jun 13 00:56:00.778395 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.778386 2552 manager.go:349] Starting recovery of all containers Jun 13 00:56:00.779659 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.779627 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 13 00:56:00.779739 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.779642 2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-107.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 13 00:56:00.788739 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.788724 2552 manager.go:354] Recovery completed Jun 13 00:56:00.792520 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.792508 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 13 00:56:00.794444 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.794421 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientMemory" Jun 13 00:56:00.794528 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.794457 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasNoDiskPressure" Jun 13 00:56:00.794528 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.794469 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientPID" Jun 13 00:56:00.794945 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.794930 2552 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 13 00:56:00.794991 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.794947 2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 13 00:56:00.794991 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.794966 2552 state_mem.go:36] "Initialized new in-memory state store" Jun 13 00:56:00.795047 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.794947 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 13 00:56:00.795626 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.795611 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientMemory" Jun 13 00:56:00.795694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.795636 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasNoDiskPressure" Jun 13 00:56:00.795694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.795646 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientPID" Jun 13 00:56:00.797114 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.797101 2552 policy_none.go:49] "None policy: Start" Jun 13 00:56:00.797172 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.797118 2552 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 13 00:56:00.797172 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.797131 2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 13 00:56:00.799167 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.799152 2552 policy_none.go:47] "Start" Jun 13 00:56:00.806154 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.806098 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-107.ec2.internal.18b87df8f23c7e8f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-107.ec2.internal,UID:ip-10-0-137-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-107.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-107.ec2.internal,},FirstTimestamp:2026-06-13 00:56:00.794443407 +0000 UTC m=+0.368372286,LastTimestamp:2026-06-13 00:56:00.794443407 +0000 UTC m=+0.368372286,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-107.ec2.internal,}" Jun 13 00:56:00.808838 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.808771 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-107.ec2.internal.18b87df8f23ccccb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-107.ec2.internal,UID:ip-10-0-137-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-107.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-107.ec2.internal,},FirstTimestamp:2026-06-13 00:56:00.794463435 +0000 UTC m=+0.368392317,LastTimestamp:2026-06-13 00:56:00.794463435 +0000 UTC m=+0.368392317,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-107.ec2.internal,}" Jun 13 00:56:00.814507 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.814492 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zf2hw" Jun 13 00:56:00.817584 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.817535 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-107.ec2.internal.18b87df8f23cf166 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-107.ec2.internal,UID:ip-10-0-137-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-107.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-107.ec2.internal,},FirstTimestamp:2026-06-13 00:56:00.794472806 +0000 UTC m=+0.368401687,LastTimestamp:2026-06-13 00:56:00.794472806 +0000 UTC m=+0.368401687,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-107.ec2.internal,}" Jun 13 00:56:00.823863 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.823817 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zf2hw" Jun 13 00:56:00.828447 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.828388 2552 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-107.ec2.internal.18b87df8f23c7e8f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-107.ec2.internal.18b87df8f23c7e8f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-107.ec2.internal,UID:ip-10-0-137-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-107.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-107.ec2.internal,},FirstTimestamp:2026-06-13 00:56:00.794443407 +0000 UTC m=+0.368372286,LastTimestamp:2026-06-13 00:56:00.795623739 +0000 UTC m=+0.369552616,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-107.ec2.internal,}" Jun 13 00:56:00.838305 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.838285 2552 manager.go:337] "Starting Device Plugin manager" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.838318 2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.838328 2552 server.go:87] "Starting device plugin registration server" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.838487 2552 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.838499 2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.838596 2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.838652 2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.838660 2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.839144 2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.839196 2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.847735 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.848740 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.848770 2552 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.848806 2552 kubelet.go:2448] "Starting kubelet main sync loop" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.848841 2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 13 00:56:00.861130 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.851022 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 13 00:56:00.938998 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.938977 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 13 00:56:00.939596 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.939581 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientMemory" Jun 13 00:56:00.939683 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.939609 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasNoDiskPressure" Jun 13 00:56:00.939683 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.939622 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientPID" Jun 13 00:56:00.939683 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.939646 2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-107.ec2.internal" Jun 13 00:56:00.949830 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.949774 2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal"] Jun 13 00:56:00.949906 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.949843 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 13 00:56:00.951101 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.951088 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientMemory" Jun 13 00:56:00.951169 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.951110 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasNoDiskPressure" Jun 13 00:56:00.951169 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.951119 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientPID" Jun 13 00:56:00.951271 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.951259 2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-107.ec2.internal" Jun 13 00:56:00.951318 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.951281 2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-107.ec2.internal\": node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:00.953892 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.953880 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 13 00:56:00.953996 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.953982 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" Jun 13 00:56:00.954040 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.954007 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 13 00:56:00.954564 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.954550 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientMemory" Jun 13 00:56:00.954630 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.954569 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientMemory" Jun 13 00:56:00.954630 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.954577 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasNoDiskPressure" Jun 13 00:56:00.954630 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.954591 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasNoDiskPressure" Jun 13 00:56:00.954630 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.954604 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientPID" Jun 13 00:56:00.954743 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.954591 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientPID" Jun 13 00:56:00.956606 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.956589 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal" Jun 13 00:56:00.956680 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.956618 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 13 00:56:00.957250 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.957234 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientMemory" Jun 13 00:56:00.957341 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.957259 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasNoDiskPressure" Jun 13 00:56:00.957341 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.957272 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeHasSufficientPID" Jun 13 00:56:00.972905 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.972891 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:00.979086 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.979067 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/86dcde8117c8f08bff9dfbde122f9160-config\") pod \"kube-apiserver-proxy-ip-10-0-137-107.ec2.internal\" (UID: \"86dcde8117c8f08bff9dfbde122f9160\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal" Jun 13 00:56:00.979159 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.979091 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d6550479296eab4eb939f8e3e3cce471-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal\" (UID: \"d6550479296eab4eb939f8e3e3cce471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" Jun 13 00:56:00.979159 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:00.979107 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6550479296eab4eb939f8e3e3cce471-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal\" (UID: \"d6550479296eab4eb939f8e3e3cce471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" Jun 13 00:56:00.992287 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.992271 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-107.ec2.internal\" not found" node="ip-10-0-137-107.ec2.internal" Jun 13 00:56:00.996431 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:00.996416 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-107.ec2.internal\" not found" node="ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.073064 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:01.073041 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:01.079314 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.079294 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d6550479296eab4eb939f8e3e3cce471-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal\" (UID: \"d6550479296eab4eb939f8e3e3cce471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.079387 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.079323 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6550479296eab4eb939f8e3e3cce471-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal\" (UID: \"d6550479296eab4eb939f8e3e3cce471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.079387 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.079346 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/86dcde8117c8f08bff9dfbde122f9160-config\") pod \"kube-apiserver-proxy-ip-10-0-137-107.ec2.internal\" (UID: \"86dcde8117c8f08bff9dfbde122f9160\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.079479 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.079392 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/86dcde8117c8f08bff9dfbde122f9160-config\") pod \"kube-apiserver-proxy-ip-10-0-137-107.ec2.internal\" (UID: \"86dcde8117c8f08bff9dfbde122f9160\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.079479 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.079398 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6550479296eab4eb939f8e3e3cce471-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal\" (UID: \"d6550479296eab4eb939f8e3e3cce471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.079479 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.079435 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d6550479296eab4eb939f8e3e3cce471-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal\" (UID: \"d6550479296eab4eb939f8e3e3cce471\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.173478 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:01.173458 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:01.274184 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:01.274147 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:01.294519 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.294498 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.300245 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.300233 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.375172 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:01.375152 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:01.475625 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:01.475606 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:01.576111 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:01.576058 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:01.676766 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:01.676742 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:01.719128 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.719109 2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 13 00:56:01.719614 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.719237 2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 13 00:56:01.777260 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:01.777242 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-107.ec2.internal\" not found" Jun 13 00:56:01.810013 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:01.809990 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86dcde8117c8f08bff9dfbde122f9160.slice/crio-b7d4d4c576fb4bdf8ebcea0ae7cc0095426e68699364573b74897f7e82e69118 WatchSource:0}: Error finding container b7d4d4c576fb4bdf8ebcea0ae7cc0095426e68699364573b74897f7e82e69118: Status 404 returned error can't find the container with id b7d4d4c576fb4bdf8ebcea0ae7cc0095426e68699364573b74897f7e82e69118 Jun 13 00:56:01.810356 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:01.810336 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6550479296eab4eb939f8e3e3cce471.slice/crio-fc21a91b606278db864e48dabc82503062575760b0718475a117588bcb5fbde0 WatchSource:0}: Error finding container fc21a91b606278db864e48dabc82503062575760b0718475a117588bcb5fbde0: Status 404 returned error can't find the container with id fc21a91b606278db864e48dabc82503062575760b0718475a117588bcb5fbde0 Jun 13 00:56:01.810688 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.810673 2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 13 00:56:01.814401 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.814381 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 13 00:56:01.830034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.829981 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-12 00:51:00 +0000 UTC" deadline="2028-02-18 09:48:21.957341746 +0000 UTC" Jun 13 00:56:01.830034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.830004 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14768h52m20.12734019s" Jun 13 00:56:01.852909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.852868 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal" event={"ID":"86dcde8117c8f08bff9dfbde122f9160","Type":"ContainerStarted","Data":"b7d4d4c576fb4bdf8ebcea0ae7cc0095426e68699364573b74897f7e82e69118"} Jun 13 00:56:01.853914 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.853897 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" event={"ID":"d6550479296eab4eb939f8e3e3cce471","Type":"ContainerStarted","Data":"fc21a91b606278db864e48dabc82503062575760b0718475a117588bcb5fbde0"} Jun 13 00:56:01.878085 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.878071 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.895380 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.895364 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 13 00:56:01.897108 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.897097 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal" Jun 13 00:56:01.906266 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:01.906253 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 13 00:56:02.038067 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.038044 2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 13 00:56:02.187794 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.187712 2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 13 00:56:02.757513 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.757477 2552 apiserver.go:52] "Watching apiserver" Jun 13 00:56:02.766997 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.766971 2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 13 00:56:02.768026 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.768005 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-slx5r","openshift-image-registry/node-ca-dvssx","openshift-multus/multus-fmrlv","openshift-multus/network-metrics-daemon-ln4ft","openshift-network-diagnostics/network-check-target-nl9fb","openshift-network-operator/iptables-alerter-wgzkq","kube-system/global-pull-secret-syncer-5sblb","kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh","openshift-dns/node-resolver-xf2jd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal","openshift-multus/multus-additional-cni-plugins-tqx2r","openshift-ovn-kubernetes/ovnkube-node-2998n","kube-system/konnectivity-agent-wntlh"] Jun 13 00:56:02.770386 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.770364 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.772652 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.772622 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.772795 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.772759 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 13 00:56:02.772933 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.772920 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pthp6\"" Jun 13 00:56:02.773112 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.773091 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 13 00:56:02.773219 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.773205 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 13 00:56:02.773272 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.773237 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 13 00:56:02.774855 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.774802 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 13 00:56:02.774855 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.774819 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:02.775007 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.774877 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:02.775007 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.774903 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 13 00:56:02.775835 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.775816 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 13 00:56:02.775835 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.775829 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 13 00:56:02.775835 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.775839 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 13 00:56:02.776034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.775849 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 13 00:56:02.776183 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.776169 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b2ggw\"" Jun 13 00:56:02.777186 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.777151 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:02.777289 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.777213 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:02.777353 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.777342 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 13 00:56:02.779288 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.779273 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:02.779375 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.779326 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:02.781546 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.781530 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:02.784086 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.783873 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.784086 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.783883 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 13 00:56:02.784086 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.783891 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l6sx7\"" Jun 13 00:56:02.784086 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.783925 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 13 00:56:02.784086 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.783886 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 13 00:56:02.786168 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.786149 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.786299 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.786276 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-646g2\"" Jun 13 00:56:02.786379 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.786308 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 13 00:56:02.786713 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.786695 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 13 00:56:02.788168 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788147 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-dbus\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:02.788255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788180 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:02.788255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788204 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-conf-dir\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788227 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m547k\" (UniqueName: \"kubernetes.io/projected/d69a9eb7-5078-49c0-a3c2-82452c082d79-kube-api-access-m547k\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788251 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-run-ovn-kubernetes\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788275 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-os-release\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788300 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-socket-dir-parent\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788324 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-run-netns\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788348 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-run-multus-certs\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788373 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85c21015-b7b2-4d3a-876d-e551b28b516f-env-overrides\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788387 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788396 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt2vv\" (UniqueName: \"kubernetes.io/projected/85c21015-b7b2-4d3a-876d-e551b28b516f-kube-api-access-zt2vv\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788428 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:02.788448 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788451 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-var-lib-cni-multus\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788487 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-run-ovn\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788526 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-node-log\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788554 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-log-socket\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788574 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d69a9eb7-5078-49c0-a3c2-82452c082d79-cni-binary-copy\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788581 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788598 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-daemon-config\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788620 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-etc-kubernetes\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788642 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788665 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-system-cni-dir\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788691 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-slash\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788722 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5k96s\"" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788722 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-run-systemd\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788806 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-cni-bin\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788830 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85c21015-b7b2-4d3a-876d-e551b28b516f-ovn-node-metrics-cert\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788841 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788852 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-kubelet-config\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788875 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8vqz\" (UniqueName: \"kubernetes.io/projected/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-kube-api-access-n8vqz\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:02.788909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788895 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-systemd-units\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788916 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-run-netns\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788938 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-run-openvswitch\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788980 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-cni-netd\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.788998 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85c21015-b7b2-4d3a-876d-e551b28b516f-ovnkube-script-lib\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789012 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-run-k8s-cni-cncf-io\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789039 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789051 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-var-lib-cni-bin\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789071 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789079 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-var-lib-kubelet\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789116 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-var-lib-openvswitch\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789136 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789154 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85c21015-b7b2-4d3a-876d-e551b28b516f-ovnkube-config\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789187 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-cni-dir\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789216 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-cnibin\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789241 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-hostroot\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789265 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-kubelet\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.789622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.789287 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-etc-openvswitch\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.791102 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.791081 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.795103 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.793441 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 13 00:56:02.795103 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.793448 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 13 00:56:02.795257 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.795147 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-j8qhz\"" Jun 13 00:56:02.795312 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.795282 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 13 00:56:02.795575 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.795375 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b4nnl\"" Jun 13 00:56:02.795575 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.795434 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 13 00:56:02.796519 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.796260 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:02.798656 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.798606 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:02.799136 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.799064 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 13 00:56:02.799136 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.799111 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qxvfs\"" Jun 13 00:56:02.799271 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.799257 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 13 00:56:02.800258 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.800240 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 13 00:56:02.800852 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.800835 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 13 00:56:02.802344 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.802301 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 13 00:56:02.802440 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.802348 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ccgc9\"" Jun 13 00:56:02.814922 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.814903 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2jnb" Jun 13 00:56:02.832453 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.832438 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2jnb" Jun 13 00:56:02.878509 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.878488 2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 13 00:56:02.889656 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889630 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-run-ovn\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.889767 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889667 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-node-log\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.889767 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889689 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d69a9eb7-5078-49c0-a3c2-82452c082d79-cni-binary-copy\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.889767 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889736 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-run-ovn\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.889767 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889754 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-node-log\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889773 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/638de231-57e4-4732-bc73-794b751a3249-cni-binary-copy\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889816 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889840 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9nfq\" (UniqueName: \"kubernetes.io/projected/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-kube-api-access-p9nfq\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889864 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-system-cni-dir\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889888 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-run-systemd\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889910 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-cni-bin\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.889922 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889933 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85c21015-b7b2-4d3a-876d-e551b28b516f-ovn-node-metrics-cert\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889956 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-kubelet-config\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889947 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-run-systemd\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.889982 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889982 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8vqz\" (UniqueName: \"kubernetes.io/projected/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-kube-api-access-n8vqz\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.889988 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-cni-bin\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890004 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-conf-dir\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890018 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-kubelet-config\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890027 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-lib-modules\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.890075 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret podName:ae75009f-6dcc-49b0-8d6a-56b6e2813f00 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:03.390034415 +0000 UTC m=+2.963963293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret") pod "global-pull-secret-syncer-5sblb" (UID: "ae75009f-6dcc-49b0-8d6a-56b6e2813f00") : object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890087 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-conf-dir\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890109 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a20fe3f-089b-49c8-a272-e2884618f537-tmp\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890138 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-cni-netd\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890159 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85c21015-b7b2-4d3a-876d-e551b28b516f-ovnkube-script-lib\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890179 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-run-k8s-cni-cncf-io\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890203 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-var-lib-kubelet\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890227 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-run-k8s-cni-cncf-io\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890237 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-cni-netd\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890252 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d69a9eb7-5078-49c0-a3c2-82452c082d79-cni-binary-copy\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890229 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-systemd\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890262 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-var-lib-kubelet\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.890496 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890294 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c318313b-4508-4376-8a9d-690bf9cc7aa4-host-slash\") pod \"iptables-alerter-wgzkq\" (UID: \"c318313b-4508-4376-8a9d-690bf9cc7aa4\") " pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890304 2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890320 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/638de231-57e4-4732-bc73-794b751a3249-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890337 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk4c7\" (UniqueName: \"kubernetes.io/projected/08c13986-e45f-4f66-a34e-0401741bdfcf-kube-api-access-xk4c7\") pod \"node-ca-dvssx\" (UID: \"08c13986-e45f-4f66-a34e-0401741bdfcf\") " pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890365 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85c21015-b7b2-4d3a-876d-e551b28b516f-ovnkube-config\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890394 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-cnibin\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890479 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-cnibin\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890504 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-hostroot\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890543 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-modprobe-d\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890566 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-sysctl-d\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890585 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-hostroot\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890589 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87376b5e-f663-4b36-affd-55da108f6af7-hosts-file\") pod \"node-resolver-xf2jd\" (UID: \"87376b5e-f663-4b36-affd-55da108f6af7\") " pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890637 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-run-netns\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890665 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-sysctl-conf\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890689 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-device-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890706 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85c21015-b7b2-4d3a-876d-e551b28b516f-ovnkube-script-lib\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890713 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt2vv\" (UniqueName: \"kubernetes.io/projected/85c21015-b7b2-4d3a-876d-e551b28b516f-kube-api-access-zt2vv\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890756 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-kubernetes\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.891255 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890774 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-run\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890729 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-run-netns\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890809 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66gth\" (UniqueName: \"kubernetes.io/projected/7a20fe3f-089b-49c8-a272-e2884618f537-kube-api-access-66gth\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890828 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85c21015-b7b2-4d3a-876d-e551b28b516f-ovnkube-config\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890841 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-log-socket\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890867 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-daemon-config\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890876 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-log-socket\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890889 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-etc-kubernetes\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890926 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7a20fe3f-089b-49c8-a272-e2884618f537-etc-tuned\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890949 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-registration-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890964 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-etc-kubernetes\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.890974 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-system-cni-dir\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891008 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-host\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891020 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-system-cni-dir\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891034 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-sys-fs\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891076 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6z9p\" (UniqueName: \"kubernetes.io/projected/638de231-57e4-4732-bc73-794b751a3249-kube-api-access-w6z9p\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891106 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-slash\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.892124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891130 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-var-lib-cni-bin\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891152 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m547k\" (UniqueName: \"kubernetes.io/projected/d69a9eb7-5078-49c0-a3c2-82452c082d79-kube-api-access-m547k\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891176 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-sys\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891192 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-var-lib-cni-bin\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891197 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08c13986-e45f-4f66-a34e-0401741bdfcf-host\") pod \"node-ca-dvssx\" (UID: \"08c13986-e45f-4f66-a34e-0401741bdfcf\") " pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891192 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-slash\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891246 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a671b759-54ea-4e80-8f31-e3e392510140-agent-certs\") pod \"konnectivity-agent-wntlh\" (UID: \"a671b759-54ea-4e80-8f31-e3e392510140\") " pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891278 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-systemd-units\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891301 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-run-netns\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891303 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-daemon-config\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891324 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-run-openvswitch\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891331 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-systemd-units\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891345 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-sysconfig\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891357 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-run-netns\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891382 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-run-openvswitch\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891393 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c318313b-4508-4376-8a9d-690bf9cc7aa4-iptables-alerter-script\") pod \"iptables-alerter-wgzkq\" (UID: \"c318313b-4508-4376-8a9d-690bf9cc7aa4\") " pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891421 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xgqd\" (UniqueName: \"kubernetes.io/projected/c318313b-4508-4376-8a9d-690bf9cc7aa4-kube-api-access-7xgqd\") pod \"iptables-alerter-wgzkq\" (UID: \"c318313b-4508-4376-8a9d-690bf9cc7aa4\") " pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:02.893034 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891445 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-var-lib-openvswitch\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891473 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891497 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-cni-dir\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891518 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891536 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-var-lib-openvswitch\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891545 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891540 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-socket-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891571 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-cni-dir\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891589 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwdsg\" (UniqueName: \"kubernetes.io/projected/87376b5e-f663-4b36-affd-55da108f6af7-kube-api-access-nwdsg\") pod \"node-resolver-xf2jd\" (UID: \"87376b5e-f663-4b36-affd-55da108f6af7\") " pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891616 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08c13986-e45f-4f66-a34e-0401741bdfcf-serviceca\") pod \"node-ca-dvssx\" (UID: \"08c13986-e45f-4f66-a34e-0401741bdfcf\") " pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891643 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-kubelet\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891667 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-etc-openvswitch\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891688 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-kubelet\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891689 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-dbus\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891723 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891747 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-etc-selinux\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891769 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87376b5e-f663-4b36-affd-55da108f6af7-tmp-dir\") pod \"node-resolver-xf2jd\" (UID: \"87376b5e-f663-4b36-affd-55da108f6af7\") " pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:02.893825 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891807 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-os-release\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891810 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-dbus\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.891814 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891848 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891845 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-etc-openvswitch\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.891876 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:03.391855896 +0000 UTC m=+2.965784761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891911 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-run-ovn-kubernetes\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891939 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-os-release\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891964 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-socket-dir-parent\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891980 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-run-multus-certs\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.891983 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85c21015-b7b2-4d3a-876d-e551b28b516f-host-run-ovn-kubernetes\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892003 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/638de231-57e4-4732-bc73-794b751a3249-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892026 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a671b759-54ea-4e80-8f31-e3e392510140-konnectivity-ca\") pod \"konnectivity-agent-wntlh\" (UID: \"a671b759-54ea-4e80-8f31-e3e392510140\") " pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892048 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85c21015-b7b2-4d3a-876d-e551b28b516f-env-overrides\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892049 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-run-multus-certs\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892061 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-os-release\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892071 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:02.894694 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892049 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-multus-socket-dir-parent\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.895487 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892098 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-var-lib-cni-multus\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.895487 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892124 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-var-lib-kubelet\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.895487 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892147 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-cnibin\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.895487 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892151 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d69a9eb7-5078-49c0-a3c2-82452c082d79-host-var-lib-cni-multus\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.895487 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.892370 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85c21015-b7b2-4d3a-876d-e551b28b516f-env-overrides\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.895487 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.894023 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85c21015-b7b2-4d3a-876d-e551b28b516f-ovn-node-metrics-cert\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.901283 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.901263 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 13 00:56:02.901382 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.901291 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 13 00:56:02.901382 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.901306 2552 projected.go:196] Error preparing data for projected volume kube-api-access-rfxkx for pod openshift-network-diagnostics/network-check-target-nl9fb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:02.901382 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:02.901368 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx podName:494eb8c5-81a4-43c5-9f45-64944e129a60 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:03.401353851 +0000 UTC m=+2.975282718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rfxkx" (UniqueName: "kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx") pod "network-check-target-nl9fb" (UID: "494eb8c5-81a4-43c5-9f45-64944e129a60") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:02.904042 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.904019 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt2vv\" (UniqueName: \"kubernetes.io/projected/85c21015-b7b2-4d3a-876d-e551b28b516f-kube-api-access-zt2vv\") pod \"ovnkube-node-2998n\" (UID: \"85c21015-b7b2-4d3a-876d-e551b28b516f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:02.904148 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.904140 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m547k\" (UniqueName: \"kubernetes.io/projected/d69a9eb7-5078-49c0-a3c2-82452c082d79-kube-api-access-m547k\") pod \"multus-fmrlv\" (UID: \"d69a9eb7-5078-49c0-a3c2-82452c082d79\") " pod="openshift-multus/multus-fmrlv" Jun 13 00:56:02.904662 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.904644 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8vqz\" (UniqueName: \"kubernetes.io/projected/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-kube-api-access-n8vqz\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:02.992424 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992390 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-cnibin\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.992608 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992434 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/638de231-57e4-4732-bc73-794b751a3249-cni-binary-copy\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.992608 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992470 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9nfq\" (UniqueName: \"kubernetes.io/projected/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-kube-api-access-p9nfq\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.992608 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992483 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-cnibin\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.992608 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992490 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-system-cni-dir\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.992608 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992534 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-system-cni-dir\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.992608 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992573 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-lib-modules\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992608 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992602 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a20fe3f-089b-49c8-a272-e2884618f537-tmp\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992636 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-systemd\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992695 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-systemd\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992718 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c318313b-4508-4376-8a9d-690bf9cc7aa4-host-slash\") pod \"iptables-alerter-wgzkq\" (UID: \"c318313b-4508-4376-8a9d-690bf9cc7aa4\") " pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992736 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/638de231-57e4-4732-bc73-794b751a3249-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992753 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk4c7\" (UniqueName: \"kubernetes.io/projected/08c13986-e45f-4f66-a34e-0401741bdfcf-kube-api-access-xk4c7\") pod \"node-ca-dvssx\" (UID: \"08c13986-e45f-4f66-a34e-0401741bdfcf\") " pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992769 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-lib-modules\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992794 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-modprobe-d\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992816 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-sysctl-d\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992840 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87376b5e-f663-4b36-affd-55da108f6af7-hosts-file\") pod \"node-resolver-xf2jd\" (UID: \"87376b5e-f663-4b36-affd-55da108f6af7\") " pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992863 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-sysctl-conf\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992885 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-device-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992909 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-kubernetes\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992930 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-run\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992953 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66gth\" (UniqueName: \"kubernetes.io/projected/7a20fe3f-089b-49c8-a272-e2884618f537-kube-api-access-66gth\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.992973 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.992981 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7a20fe3f-089b-49c8-a272-e2884618f537-etc-tuned\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993005 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-registration-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993019 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87376b5e-f663-4b36-affd-55da108f6af7-hosts-file\") pod \"node-resolver-xf2jd\" (UID: \"87376b5e-f663-4b36-affd-55da108f6af7\") " pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993060 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/638de231-57e4-4732-bc73-794b751a3249-cni-binary-copy\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993074 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-host\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993127 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-run\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993132 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-device-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993181 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-kubernetes\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993265 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/638de231-57e4-4732-bc73-794b751a3249-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993315 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-modprobe-d\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993328 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c318313b-4508-4376-8a9d-690bf9cc7aa4-host-slash\") pod \"iptables-alerter-wgzkq\" (UID: \"c318313b-4508-4376-8a9d-690bf9cc7aa4\") " pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993026 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-host\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993370 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-sys-fs\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993395 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6z9p\" (UniqueName: \"kubernetes.io/projected/638de231-57e4-4732-bc73-794b751a3249-kube-api-access-w6z9p\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993423 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-sys\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.993612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993618 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08c13986-e45f-4f66-a34e-0401741bdfcf-host\") pod \"node-ca-dvssx\" (UID: \"08c13986-e45f-4f66-a34e-0401741bdfcf\") " pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993643 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a671b759-54ea-4e80-8f31-e3e392510140-agent-certs\") pod \"konnectivity-agent-wntlh\" (UID: \"a671b759-54ea-4e80-8f31-e3e392510140\") " pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993671 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-sysconfig\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993683 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08c13986-e45f-4f66-a34e-0401741bdfcf-host\") pod \"node-ca-dvssx\" (UID: \"08c13986-e45f-4f66-a34e-0401741bdfcf\") " pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993694 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c318313b-4508-4376-8a9d-690bf9cc7aa4-iptables-alerter-script\") pod \"iptables-alerter-wgzkq\" (UID: \"c318313b-4508-4376-8a9d-690bf9cc7aa4\") " pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993478 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-sysctl-conf\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993471 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-registration-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993506 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-sys-fs\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993430 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-sysctl-d\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993538 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-sys\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993734 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xgqd\" (UniqueName: \"kubernetes.io/projected/c318313b-4508-4376-8a9d-690bf9cc7aa4-kube-api-access-7xgqd\") pod \"iptables-alerter-wgzkq\" (UID: \"c318313b-4508-4376-8a9d-690bf9cc7aa4\") " pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993749 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-etc-sysconfig\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993851 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993890 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-socket-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993913 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwdsg\" (UniqueName: \"kubernetes.io/projected/87376b5e-f663-4b36-affd-55da108f6af7-kube-api-access-nwdsg\") pod \"node-resolver-xf2jd\" (UID: \"87376b5e-f663-4b36-affd-55da108f6af7\") " pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993937 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08c13986-e45f-4f66-a34e-0401741bdfcf-serviceca\") pod \"node-ca-dvssx\" (UID: \"08c13986-e45f-4f66-a34e-0401741bdfcf\") " pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.993998 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-etc-selinux\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.994308 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994026 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87376b5e-f663-4b36-affd-55da108f6af7-tmp-dir\") pod \"node-resolver-xf2jd\" (UID: \"87376b5e-f663-4b36-affd-55da108f6af7\") " pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994050 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-os-release\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994053 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994072 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994105 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/638de231-57e4-4732-bc73-794b751a3249-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994130 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a671b759-54ea-4e80-8f31-e3e392510140-konnectivity-ca\") pod \"konnectivity-agent-wntlh\" (UID: \"a671b759-54ea-4e80-8f31-e3e392510140\") " pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994164 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-var-lib-kubelet\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994236 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a20fe3f-089b-49c8-a272-e2884618f537-var-lib-kubelet\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994284 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-etc-selinux\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994373 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-socket-dir\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994438 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994441 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08c13986-e45f-4f66-a34e-0401741bdfcf-serviceca\") pod \"node-ca-dvssx\" (UID: \"08c13986-e45f-4f66-a34e-0401741bdfcf\") " pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994532 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/638de231-57e4-4732-bc73-794b751a3249-os-release\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994568 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c318313b-4508-4376-8a9d-690bf9cc7aa4-iptables-alerter-script\") pod \"iptables-alerter-wgzkq\" (UID: \"c318313b-4508-4376-8a9d-690bf9cc7aa4\") " pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994647 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87376b5e-f663-4b36-affd-55da108f6af7-tmp-dir\") pod \"node-resolver-xf2jd\" (UID: \"87376b5e-f663-4b36-affd-55da108f6af7\") " pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.994968 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/638de231-57e4-4732-bc73-794b751a3249-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:02.995120 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.995061 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a671b759-54ea-4e80-8f31-e3e392510140-konnectivity-ca\") pod \"konnectivity-agent-wntlh\" (UID: \"a671b759-54ea-4e80-8f31-e3e392510140\") " pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:02.995917 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.995739 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7a20fe3f-089b-49c8-a272-e2884618f537-etc-tuned\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.995917 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.995771 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a20fe3f-089b-49c8-a272-e2884618f537-tmp\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:02.996157 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:02.996140 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a671b759-54ea-4e80-8f31-e3e392510140-agent-certs\") pod \"konnectivity-agent-wntlh\" (UID: \"a671b759-54ea-4e80-8f31-e3e392510140\") " pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:03.001199 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.001157 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk4c7\" (UniqueName: \"kubernetes.io/projected/08c13986-e45f-4f66-a34e-0401741bdfcf-kube-api-access-xk4c7\") pod \"node-ca-dvssx\" (UID: \"08c13986-e45f-4f66-a34e-0401741bdfcf\") " pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:03.001764 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.001741 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9nfq\" (UniqueName: \"kubernetes.io/projected/18b28079-82d6-4ec7-a9fe-5b8e8d52c703-kube-api-access-p9nfq\") pod \"aws-ebs-csi-driver-node-p5nxh\" (UID: \"18b28079-82d6-4ec7-a9fe-5b8e8d52c703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:03.002155 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.002135 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66gth\" (UniqueName: \"kubernetes.io/projected/7a20fe3f-089b-49c8-a272-e2884618f537-kube-api-access-66gth\") pod \"tuned-slx5r\" (UID: \"7a20fe3f-089b-49c8-a272-e2884618f537\") " pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:03.002519 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.002499 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6z9p\" (UniqueName: \"kubernetes.io/projected/638de231-57e4-4732-bc73-794b751a3249-kube-api-access-w6z9p\") pod \"multus-additional-cni-plugins-tqx2r\" (UID: \"638de231-57e4-4732-bc73-794b751a3249\") " pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:03.002728 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.002713 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xgqd\" (UniqueName: \"kubernetes.io/projected/c318313b-4508-4376-8a9d-690bf9cc7aa4-kube-api-access-7xgqd\") pod \"iptables-alerter-wgzkq\" (UID: \"c318313b-4508-4376-8a9d-690bf9cc7aa4\") " pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:03.002762 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.002740 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwdsg\" (UniqueName: \"kubernetes.io/projected/87376b5e-f663-4b36-affd-55da108f6af7-kube-api-access-nwdsg\") pod \"node-resolver-xf2jd\" (UID: \"87376b5e-f663-4b36-affd-55da108f6af7\") " pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:03.081074 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.080998 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fmrlv" Jun 13 00:56:03.087471 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.087448 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:03.097238 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.097220 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgzkq" Jun 13 00:56:03.104371 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.104348 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-slx5r" Jun 13 00:56:03.109881 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.109860 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" Jun 13 00:56:03.115393 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.115378 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xf2jd" Jun 13 00:56:03.121861 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.121843 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" Jun 13 00:56:03.128307 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.128292 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dvssx" Jun 13 00:56:03.132863 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.132843 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:03.223127 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.223095 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 13 00:56:03.397909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.397831 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:03.397909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.397901 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:03.398119 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:03.397985 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:03.398119 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:03.397998 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:03.398119 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:03.398061 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret podName:ae75009f-6dcc-49b0-8d6a-56b6e2813f00 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:04.398041857 +0000 UTC m=+3.971970739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret") pod "global-pull-secret-syncer-5sblb" (UID: "ae75009f-6dcc-49b0-8d6a-56b6e2813f00") : object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:03.398119 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:03.398081 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:04.398072554 +0000 UTC m=+3.972001419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:03.499143 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.499097 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:03.499314 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:03.499267 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 13 00:56:03.499314 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:03.499293 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 13 00:56:03.499314 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:03.499306 2552 projected.go:196] Error preparing data for projected volume kube-api-access-rfxkx for pod openshift-network-diagnostics/network-check-target-nl9fb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:03.499468 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:03.499364 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx podName:494eb8c5-81a4-43c5-9f45-64944e129a60 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:04.499351739 +0000 UTC m=+4.073280604 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfxkx" (UniqueName: "kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx") pod "network-check-target-nl9fb" (UID: "494eb8c5-81a4-43c5-9f45-64944e129a60") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:03.616307 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:03.616254 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69a9eb7_5078_49c0_a3c2_82452c082d79.slice/crio-8b188d0be3ede2fb1c3a424f7ba38659fda6109f7e538b93ccc461d361354752 WatchSource:0}: Error finding container 8b188d0be3ede2fb1c3a424f7ba38659fda6109f7e538b93ccc461d361354752: Status 404 returned error can't find the container with id 8b188d0be3ede2fb1c3a424f7ba38659fda6109f7e538b93ccc461d361354752 Jun 13 00:56:03.617556 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:03.617531 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c13986_e45f_4f66_a34e_0401741bdfcf.slice/crio-9b34d84f2714ec50257f55439ef335e52a23f96c25e4bdc14a07830cd9dc2a7e WatchSource:0}: Error finding container 9b34d84f2714ec50257f55439ef335e52a23f96c25e4bdc14a07830cd9dc2a7e: Status 404 returned error can't find the container with id 9b34d84f2714ec50257f55439ef335e52a23f96c25e4bdc14a07830cd9dc2a7e Jun 13 00:56:03.621477 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:03.621456 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a20fe3f_089b_49c8_a272_e2884618f537.slice/crio-363b2514994c570a8cd25612b416f908462d0a9f0a507b17f782d6b933d2f6db WatchSource:0}: Error finding container 363b2514994c570a8cd25612b416f908462d0a9f0a507b17f782d6b933d2f6db: Status 404 returned error can't find the container with id 363b2514994c570a8cd25612b416f908462d0a9f0a507b17f782d6b933d2f6db Jun 13 00:56:03.641292 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:03.641273 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod638de231_57e4_4732_bc73_794b751a3249.slice/crio-1db00bce3a081925f38c71b5790a621110f2bbd361db0c5f970cacc2a4c3f010 WatchSource:0}: Error finding container 1db00bce3a081925f38c71b5790a621110f2bbd361db0c5f970cacc2a4c3f010: Status 404 returned error can't find the container with id 1db00bce3a081925f38c71b5790a621110f2bbd361db0c5f970cacc2a4c3f010 Jun 13 00:56:03.642645 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:03.642535 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b28079_82d6_4ec7_a9fe_5b8e8d52c703.slice/crio-41bdd68e641f83fd1aa2b49675b58378d4503e892a96e92209a31fb396cf1def WatchSource:0}: Error finding container 41bdd68e641f83fd1aa2b49675b58378d4503e892a96e92209a31fb396cf1def: Status 404 returned error can't find the container with id 41bdd68e641f83fd1aa2b49675b58378d4503e892a96e92209a31fb396cf1def Jun 13 00:56:03.643229 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:03.643207 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc318313b_4508_4376_8a9d_690bf9cc7aa4.slice/crio-ae0ae45cac2025e6160de2a1b9e153810d33ad01a6ca2e7405d3e5c5202fdb3b WatchSource:0}: Error finding container ae0ae45cac2025e6160de2a1b9e153810d33ad01a6ca2e7405d3e5c5202fdb3b: Status 404 returned error can't find the container with id ae0ae45cac2025e6160de2a1b9e153810d33ad01a6ca2e7405d3e5c5202fdb3b Jun 13 00:56:03.644608 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:03.644587 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda671b759_54ea_4e80_8f31_e3e392510140.slice/crio-53ec6598bfa843fd5bbbaed175cfca181b38aa6ada203b86bb17641337f57e1a WatchSource:0}: Error finding container 53ec6598bfa843fd5bbbaed175cfca181b38aa6ada203b86bb17641337f57e1a: Status 404 returned error can't find the container with id 53ec6598bfa843fd5bbbaed175cfca181b38aa6ada203b86bb17641337f57e1a Jun 13 00:56:03.645095 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:03.645072 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87376b5e_f663_4b36_affd_55da108f6af7.slice/crio-d294a125c9b2a7550ff053d1179c1736b787c310c00e065ce099803be2c87234 WatchSource:0}: Error finding container d294a125c9b2a7550ff053d1179c1736b787c310c00e065ce099803be2c87234: Status 404 returned error can't find the container with id d294a125c9b2a7550ff053d1179c1736b787c310c00e065ce099803be2c87234 Jun 13 00:56:03.645735 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:03.645712 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c21015_b7b2_4d3a_876d_e551b28b516f.slice/crio-7a41edb78461b0a86f85bbb44bc3a2feff0d6b51efa4cb52efe696c529a34953 WatchSource:0}: Error finding container 7a41edb78461b0a86f85bbb44bc3a2feff0d6b51efa4cb52efe696c529a34953: Status 404 returned error can't find the container with id 7a41edb78461b0a86f85bbb44bc3a2feff0d6b51efa4cb52efe696c529a34953 Jun 13 00:56:03.835509 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.835408 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-12 00:51:02 +0000 UTC" deadline="2028-03-15 20:13:07.573499862 +0000 UTC" Jun 13 00:56:03.835509 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.835507 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15403h17m3.737999376s" Jun 13 00:56:03.859017 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.858983 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal" event={"ID":"86dcde8117c8f08bff9dfbde122f9160","Type":"ContainerStarted","Data":"d09f5df5e4f79aa6a7d26e12d8902b35cf7027f0f578ef793dd212ca00a447b6"} Jun 13 00:56:03.860094 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.860072 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerStarted","Data":"7a41edb78461b0a86f85bbb44bc3a2feff0d6b51efa4cb52efe696c529a34953"} Jun 13 00:56:03.861173 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.861151 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgzkq" event={"ID":"c318313b-4508-4376-8a9d-690bf9cc7aa4","Type":"ContainerStarted","Data":"ae0ae45cac2025e6160de2a1b9e153810d33ad01a6ca2e7405d3e5c5202fdb3b"} Jun 13 00:56:03.862220 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.862193 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" event={"ID":"18b28079-82d6-4ec7-a9fe-5b8e8d52c703","Type":"ContainerStarted","Data":"41bdd68e641f83fd1aa2b49675b58378d4503e892a96e92209a31fb396cf1def"} Jun 13 00:56:03.863375 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.863354 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" event={"ID":"638de231-57e4-4732-bc73-794b751a3249","Type":"ContainerStarted","Data":"1db00bce3a081925f38c71b5790a621110f2bbd361db0c5f970cacc2a4c3f010"} Jun 13 00:56:03.864582 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.864553 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fmrlv" event={"ID":"d69a9eb7-5078-49c0-a3c2-82452c082d79","Type":"ContainerStarted","Data":"8b188d0be3ede2fb1c3a424f7ba38659fda6109f7e538b93ccc461d361354752"} Jun 13 00:56:03.865562 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.865543 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xf2jd" event={"ID":"87376b5e-f663-4b36-affd-55da108f6af7","Type":"ContainerStarted","Data":"d294a125c9b2a7550ff053d1179c1736b787c310c00e065ce099803be2c87234"} Jun 13 00:56:03.866611 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.866594 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wntlh" event={"ID":"a671b759-54ea-4e80-8f31-e3e392510140","Type":"ContainerStarted","Data":"53ec6598bfa843fd5bbbaed175cfca181b38aa6ada203b86bb17641337f57e1a"} Jun 13 00:56:03.867492 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.867475 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-slx5r" event={"ID":"7a20fe3f-089b-49c8-a272-e2884618f537","Type":"ContainerStarted","Data":"363b2514994c570a8cd25612b416f908462d0a9f0a507b17f782d6b933d2f6db"} Jun 13 00:56:03.868635 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.868614 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dvssx" event={"ID":"08c13986-e45f-4f66-a34e-0401741bdfcf","Type":"ContainerStarted","Data":"9b34d84f2714ec50257f55439ef335e52a23f96c25e4bdc14a07830cd9dc2a7e"} Jun 13 00:56:03.875173 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:03.875123 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-107.ec2.internal" podStartSLOduration=2.875111082 podStartE2EDuration="2.875111082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:03.874741994 +0000 UTC m=+3.448670881" watchObservedRunningTime="2026-06-13 00:56:03.875111082 +0000 UTC m=+3.449039969" Jun 13 00:56:04.406725 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.405999 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:04.406725 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.406060 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:04.406725 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.406179 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:04.406725 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.406233 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret podName:ae75009f-6dcc-49b0-8d6a-56b6e2813f00 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:06.406216676 +0000 UTC m=+5.980145548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret") pod "global-pull-secret-syncer-5sblb" (UID: "ae75009f-6dcc-49b0-8d6a-56b6e2813f00") : object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:04.406725 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.406597 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:04.406725 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.406645 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:06.406630833 +0000 UTC m=+5.980559702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:04.506577 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.506541 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:04.506749 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.506693 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 13 00:56:04.506749 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.506715 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 13 00:56:04.506749 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.506726 2552 projected.go:196] Error preparing data for projected volume kube-api-access-rfxkx for pod openshift-network-diagnostics/network-check-target-nl9fb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:04.506937 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.506796 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx podName:494eb8c5-81a4-43c5-9f45-64944e129a60 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:06.506761914 +0000 UTC m=+6.080690780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfxkx" (UniqueName: "kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx") pod "network-check-target-nl9fb" (UID: "494eb8c5-81a4-43c5-9f45-64944e129a60") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:04.836470 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.836370 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-12 00:51:02 +0000 UTC" deadline="2027-12-27 16:52:11.33423812 +0000 UTC" Jun 13 00:56:04.836470 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.836416 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13503h56m6.497826279s" Jun 13 00:56:04.850562 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.849938 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:04.850562 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.850058 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:04.850562 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.850264 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:04.850562 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.850396 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:04.850562 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.850435 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:04.850562 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:04.850526 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:04.878750 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.878726 2552 generic.go:356] "Generic (PLEG): container finished" podID="d6550479296eab4eb939f8e3e3cce471" containerID="b50cac8f0e8ee449cd27e3abf97f2500c4be4ef0218f9a64b04a2bce40606a2e" exitCode=0 Jun 13 00:56:04.879399 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:04.879379 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" event={"ID":"d6550479296eab4eb939f8e3e3cce471","Type":"ContainerDied","Data":"b50cac8f0e8ee449cd27e3abf97f2500c4be4ef0218f9a64b04a2bce40606a2e"} Jun 13 00:56:05.898732 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:05.898698 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" event={"ID":"d6550479296eab4eb939f8e3e3cce471","Type":"ContainerStarted","Data":"2dae68276f5c08d59b09d1031a0b1ee8554d3d6ca7406ee796f4f317fbcf435d"} Jun 13 00:56:06.418567 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:06.418535 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:06.418720 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:06.418598 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:06.418720 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.418716 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:06.418857 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.418771 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret podName:ae75009f-6dcc-49b0-8d6a-56b6e2813f00 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:10.418754985 +0000 UTC m=+9.992683854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret") pod "global-pull-secret-syncer-5sblb" (UID: "ae75009f-6dcc-49b0-8d6a-56b6e2813f00") : object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:06.418916 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.418861 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:06.418916 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.418896 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:10.418884299 +0000 UTC m=+9.992813166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:06.518924 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:06.518892 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:06.519104 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.519088 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 13 00:56:06.519169 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.519109 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 13 00:56:06.519169 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.519122 2552 projected.go:196] Error preparing data for projected volume kube-api-access-rfxkx for pod openshift-network-diagnostics/network-check-target-nl9fb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:06.519264 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.519179 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx podName:494eb8c5-81a4-43c5-9f45-64944e129a60 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:10.519162398 +0000 UTC m=+10.093091269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfxkx" (UniqueName: "kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx") pod "network-check-target-nl9fb" (UID: "494eb8c5-81a4-43c5-9f45-64944e129a60") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:06.850612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:06.850527 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:06.850772 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.850676 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:06.851178 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:06.851080 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:06.851280 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.851215 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:06.851280 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:06.851273 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:06.851429 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:06.851364 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:08.849156 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:08.849121 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:08.849598 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:08.849245 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:08.849598 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:08.849127 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:08.849598 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:08.849505 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:08.849960 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:08.849824 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:08.849960 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:08.849926 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:10.452001 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:10.451320 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:10.452001 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:10.451384 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:10.452001 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.451496 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:10.452001 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.451551 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret podName:ae75009f-6dcc-49b0-8d6a-56b6e2813f00 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:18.451534086 +0000 UTC m=+18.025462952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret") pod "global-pull-secret-syncer-5sblb" (UID: "ae75009f-6dcc-49b0-8d6a-56b6e2813f00") : object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:10.452001 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.451920 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:10.452001 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.451969 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:18.451954681 +0000 UTC m=+18.025883549 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:10.552070 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:10.552037 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:10.552236 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.552201 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 13 00:56:10.552236 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.552224 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 13 00:56:10.552350 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.552238 2552 projected.go:196] Error preparing data for projected volume kube-api-access-rfxkx for pod openshift-network-diagnostics/network-check-target-nl9fb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:10.552350 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.552291 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx podName:494eb8c5-81a4-43c5-9f45-64944e129a60 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:18.552274293 +0000 UTC m=+18.126203172 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfxkx" (UniqueName: "kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx") pod "network-check-target-nl9fb" (UID: "494eb8c5-81a4-43c5-9f45-64944e129a60") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:10.851038 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:10.850963 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:10.851190 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.851058 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:10.851434 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:10.851419 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:10.851560 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.851517 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:10.851667 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:10.851633 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:10.851708 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:10.851687 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:12.849686 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:12.849300 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:12.849686 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:12.849318 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:12.849686 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:12.849419 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:12.849686 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:12.849475 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:12.849686 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:12.849513 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:12.849686 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:12.849604 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:14.849461 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:14.849426 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:14.849891 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:14.849441 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:14.849891 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:14.849548 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:14.849891 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:14.849547 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:14.849891 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:14.849621 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:14.849891 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:14.849692 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:16.849010 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:16.848978 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:16.849010 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:16.849009 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:16.849450 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:16.848979 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:16.849450 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:16.849099 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:16.849450 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:16.849173 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:16.849450 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:16.849281 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:18.513768 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:18.513726 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:18.514230 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:18.513796 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:18.514230 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.513886 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:18.514230 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.513890 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:18.514230 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.513947 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret podName:ae75009f-6dcc-49b0-8d6a-56b6e2813f00 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:34.513929205 +0000 UTC m=+34.087858089 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret") pod "global-pull-secret-syncer-5sblb" (UID: "ae75009f-6dcc-49b0-8d6a-56b6e2813f00") : object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:18.514230 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.513968 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:34.51395823 +0000 UTC m=+34.087887095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:18.614877 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:18.614846 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:18.615044 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.615015 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 13 00:56:18.615044 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.615042 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 13 00:56:18.615142 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.615056 2552 projected.go:196] Error preparing data for projected volume kube-api-access-rfxkx for pod openshift-network-diagnostics/network-check-target-nl9fb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:18.615142 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.615111 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx podName:494eb8c5-81a4-43c5-9f45-64944e129a60 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:34.615096416 +0000 UTC m=+34.189025281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfxkx" (UniqueName: "kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx") pod "network-check-target-nl9fb" (UID: "494eb8c5-81a4-43c5-9f45-64944e129a60") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:18.850047 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:18.849974 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:18.850047 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:18.850003 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:18.850047 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:18.850012 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:18.850304 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.850088 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:18.850304 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.850196 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:18.850304 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:18.850270 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:20.852892 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:20.852863 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:20.853293 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:20.852959 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:20.853293 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:20.852987 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:20.853293 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:20.853095 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:20.853293 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:20.853161 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:20.853293 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:20.853238 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:21.930929 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.930891 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xf2jd" event={"ID":"87376b5e-f663-4b36-affd-55da108f6af7","Type":"ContainerStarted","Data":"c297d7a6f859cfb40bc69a7cae42e861d3f8c397289651ed5eec96205a86beac"} Jun 13 00:56:21.932447 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.932420 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wntlh" event={"ID":"a671b759-54ea-4e80-8f31-e3e392510140","Type":"ContainerStarted","Data":"fdd7c0be405f9c72f7409c957f2fa2a3a3cfb61f179b34305b4731e10b7144f3"} Jun 13 00:56:21.933726 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.933700 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-slx5r" event={"ID":"7a20fe3f-089b-49c8-a272-e2884618f537","Type":"ContainerStarted","Data":"f1c6172e9ece1fa24191abe78126f5433cca4ba7a77df0031172fd9ac818fea1"} Jun 13 00:56:21.935124 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.935088 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dvssx" event={"ID":"08c13986-e45f-4f66-a34e-0401741bdfcf","Type":"ContainerStarted","Data":"7915f28a93b47dd89afc26c378b99f72b16d0ef69abf7362e2402e4154c5d678"} Jun 13 00:56:21.936887 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.936871 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 00:56:21.937252 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.937231 2552 generic.go:356] "Generic (PLEG): container finished" podID="85c21015-b7b2-4d3a-876d-e551b28b516f" containerID="a2326297a6c391a3d54bc872fa5bddc3230786a36aa9103918ed58075d23de9c" exitCode=1 Jun 13 00:56:21.937360 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.937293 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerStarted","Data":"dd9f396cb26dc371ae9fc7312d339a43f9e4c6641215eaef7558c41848f9e6b8"} Jun 13 00:56:21.937421 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.937369 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerDied","Data":"a2326297a6c391a3d54bc872fa5bddc3230786a36aa9103918ed58075d23de9c"} Jun 13 00:56:21.937475 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.937425 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerStarted","Data":"bd82b3306c3cbce81f9941bbcd31a7b4dbb5892f0952681a3d285ae75e5e2721"} Jun 13 00:56:21.938664 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.938639 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" event={"ID":"18b28079-82d6-4ec7-a9fe-5b8e8d52c703","Type":"ContainerStarted","Data":"5e0be0ed3f46b48c038f18432cfd942aa025ba5cbec883e8c5571d818839c623"} Jun 13 00:56:21.940230 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.940209 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" event={"ID":"638de231-57e4-4732-bc73-794b751a3249","Type":"ContainerStarted","Data":"725a53f6245f12451df3a2f2110d35df7b4e92b9dea6476d23f6fab4b4774b02"} Jun 13 00:56:21.941557 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.941536 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fmrlv" event={"ID":"d69a9eb7-5078-49c0-a3c2-82452c082d79","Type":"ContainerStarted","Data":"7725f56a52610706b534a46d482654f26d815c68f525b45cc939b407443565e5"} Jun 13 00:56:21.949046 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.949005 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xf2jd" podStartSLOduration=3.354663179 podStartE2EDuration="20.948992212s" totalImagesPullingTime="17.594329033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:21.948865942 +0000 UTC m=+21.522794853" watchObservedRunningTime="2026-06-13 00:56:21.948992212 +0000 UTC m=+21.522921099" Jun 13 00:56:21.949149 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.949089 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-107.ec2.internal" podStartSLOduration=20.949081941 podStartE2EDuration="20.949081941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:05.915488807 +0000 UTC m=+5.489417713" watchObservedRunningTime="2026-06-13 00:56:21.949081941 +0000 UTC m=+21.523010840" Jun 13 00:56:21.979376 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:21.979335 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dvssx" podStartSLOduration=3.359799571 podStartE2EDuration="20.979320705s" totalImagesPullingTime="17.619521134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:21.978911331 +0000 UTC m=+21.552840220" watchObservedRunningTime="2026-06-13 00:56:21.979320705 +0000 UTC m=+21.553249591" Jun 13 00:56:22.013818 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.013760 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fmrlv" podStartSLOduration=4.368305316 podStartE2EDuration="22.013746998s" totalImagesPullingTime="17.645441682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:22.013670676 +0000 UTC m=+21.587599574" watchObservedRunningTime="2026-06-13 00:56:22.013746998 +0000 UTC m=+21.587675884" Jun 13 00:56:22.078386 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.078337 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wntlh" podStartSLOduration=3.528632655 podStartE2EDuration="21.078317497s" totalImagesPullingTime="17.549684842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:22.056612371 +0000 UTC m=+21.630541258" watchObservedRunningTime="2026-06-13 00:56:22.078317497 +0000 UTC m=+21.652246386" Jun 13 00:56:22.078464 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.078440 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-slx5r" podStartSLOduration=3.481374873 podStartE2EDuration="21.078431895s" totalImagesPullingTime="17.597057022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:22.077958821 +0000 UTC m=+21.651887709" watchObservedRunningTime="2026-06-13 00:56:22.078431895 +0000 UTC m=+21.652360785" Jun 13 00:56:22.849626 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.849605 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:22.849728 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.849610 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:22.849728 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:22.849696 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:22.849728 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.849718 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:22.849855 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:22.849811 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:22.849909 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:22.849883 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:22.946992 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.946966 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 00:56:22.947410 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.947386 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerStarted","Data":"7293909079cc3377acc3b376cc012e771f521b0d9b84187c8f39c5505b2c2de0"} Jun 13 00:56:22.947512 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.947493 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerStarted","Data":"f9922e05a54fec8cd0448e43d033b28da1da1138300c74ba76b1a16ca11550b6"} Jun 13 00:56:22.947577 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.947566 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerStarted","Data":"4f764383bd5caecc7a7e87a645482c2e71a4f62486a924e1981ef17602b893be"} Jun 13 00:56:22.949135 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.949111 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgzkq" event={"ID":"c318313b-4508-4376-8a9d-690bf9cc7aa4","Type":"ContainerStarted","Data":"afe34bbfcd54f8cd05866fc17ae89dd386ba03f5debcefb16de04eb9dd6dd788"} Jun 13 00:56:22.951200 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.951173 2552 generic.go:356] "Generic (PLEG): container finished" podID="638de231-57e4-4732-bc73-794b751a3249" containerID="725a53f6245f12451df3a2f2110d35df7b4e92b9dea6476d23f6fab4b4774b02" exitCode=0 Jun 13 00:56:22.951294 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.951277 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" event={"ID":"638de231-57e4-4732-bc73-794b751a3249","Type":"ContainerDied","Data":"725a53f6245f12451df3a2f2110d35df7b4e92b9dea6476d23f6fab4b4774b02"} Jun 13 00:56:22.986226 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:22.986190 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wgzkq" podStartSLOduration=4.395489238 podStartE2EDuration="21.986176989s" totalImagesPullingTime="17.590687751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:22.964164656 +0000 UTC m=+22.538093542" watchObservedRunningTime="2026-06-13 00:56:22.986176989 +0000 UTC m=+22.560105877" Jun 13 00:56:23.018553 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:23.018536 2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 13 00:56:23.852549 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:23.852359 2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-13T00:56:23.018550257Z","UUID":"c425f0ed-ea4b-46b7-aaac-be76999c4a15","Handler":null,"Name":"","Endpoint":""} Jun 13 00:56:23.853748 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:23.853728 2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 13 00:56:23.853848 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:23.853762 2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 13 00:56:23.955832 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:23.955802 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" event={"ID":"18b28079-82d6-4ec7-a9fe-5b8e8d52c703","Type":"ContainerStarted","Data":"aa29f6c35edbfe5ecbc48a50d41978ee5e03da4e6343d28f32ea21a974deb6a7"} Jun 13 00:56:23.956190 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:23.955864 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" event={"ID":"18b28079-82d6-4ec7-a9fe-5b8e8d52c703","Type":"ContainerStarted","Data":"dc3e53408f9a4c5203369ab871a54325fb8c3981080f32a7aea5a0b05ad89162"} Jun 13 00:56:23.975384 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:23.975344 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p5nxh" podStartSLOduration=3.454820938 podStartE2EDuration="22.975331767s" totalImagesPullingTime="19.520510829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:23.975067491 +0000 UTC m=+23.548996378" watchObservedRunningTime="2026-06-13 00:56:23.975331767 +0000 UTC m=+23.549260649" Jun 13 00:56:24.849354 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:24.849318 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:24.849507 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:24.849437 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:24.849507 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:24.849325 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:24.849628 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:24.849516 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:24.849628 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:24.849318 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:24.849628 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:24.849580 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:24.961013 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:24.960990 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 00:56:24.961615 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:24.961347 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerStarted","Data":"b54e60ab57f00bf28ce7c3b3a9c84958bcf56f18301a7d7d450c6485f21214e7"} Jun 13 00:56:25.547573 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:25.547546 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:25.548404 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:25.548379 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:26.849711 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:26.849655 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:26.850161 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:26.849655 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:26.850161 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:26.849810 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:26.850161 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:26.849821 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:26.850161 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:26.849925 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:26.850161 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:26.850003 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:27.970369 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:27.970199 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 00:56:27.970924 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:27.970722 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerStarted","Data":"ff0dbb5c823be7e85b7fcd2d1f07f1bab7d8bb3580b611c28fc873ec794c4659"} Jun 13 00:56:27.970990 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:27.970944 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:27.971138 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:27.971113 2552 scope.go:117] "RemoveContainer" containerID="a2326297a6c391a3d54bc872fa5bddc3230786a36aa9103918ed58075d23de9c" Jun 13 00:56:27.972558 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:27.972536 2552 generic.go:356] "Generic (PLEG): container finished" podID="638de231-57e4-4732-bc73-794b751a3249" containerID="a2e9cd9ef872b66e70c3a5a442733da02914c4ebfc38f194c2eb359c1382ec08" exitCode=0 Jun 13 00:56:27.972641 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:27.972577 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" event={"ID":"638de231-57e4-4732-bc73-794b751a3249","Type":"ContainerDied","Data":"a2e9cd9ef872b66e70c3a5a442733da02914c4ebfc38f194c2eb359c1382ec08"} Jun 13 00:56:27.986563 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:27.986547 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:28.147534 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.147514 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:28.849573 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.849542 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:28.849703 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.849553 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:28.849809 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:28.849666 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:28.849809 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.849676 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:28.849958 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:28.849815 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:28.849958 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:28.849901 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:28.977672 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.977618 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 00:56:28.978073 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.978051 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" event={"ID":"85c21015-b7b2-4d3a-876d-e551b28b516f","Type":"ContainerStarted","Data":"3b1a07d0e6edae76b5bb117147af185e3b69ee9840f3406b043a4ee43715ef4f"} Jun 13 00:56:28.978390 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.978352 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:28.980089 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.980065 2552 generic.go:356] "Generic (PLEG): container finished" podID="638de231-57e4-4732-bc73-794b751a3249" containerID="daade45c5c90aebd5985ba095968ceddef6af77d232e51db9d88b043cbe29a89" exitCode=0 Jun 13 00:56:28.980170 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.980114 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" event={"ID":"638de231-57e4-4732-bc73-794b751a3249","Type":"ContainerDied","Data":"daade45c5c90aebd5985ba095968ceddef6af77d232e51db9d88b043cbe29a89"} Jun 13 00:56:28.992677 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:28.992659 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:56:29.007859 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:29.007813 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" podStartSLOduration=11.098555511 podStartE2EDuration="29.007797939s" totalImagesPullingTime="17.909242428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:29.006317265 +0000 UTC m=+28.580246152" watchObservedRunningTime="2026-06-13 00:56:29.007797939 +0000 UTC m=+28.581726824" Jun 13 00:56:29.095753 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:29.095730 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ln4ft"] Jun 13 00:56:29.095890 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:29.095875 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:29.095996 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:29.095977 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:29.098492 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:29.098465 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nl9fb"] Jun 13 00:56:29.098592 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:29.098551 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:29.098636 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:29.098620 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:29.099038 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:29.099018 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5sblb"] Jun 13 00:56:29.099132 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:29.099102 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:29.099210 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:29.099194 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:29.984596 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:29.984424 2552 generic.go:356] "Generic (PLEG): container finished" podID="638de231-57e4-4732-bc73-794b751a3249" containerID="80dfff63d9e4a012a720f0098428629ec825a8f6ffc2602c215aeed0b85fbcf0" exitCode=0 Jun 13 00:56:29.984965 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:29.984504 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" event={"ID":"638de231-57e4-4732-bc73-794b751a3249","Type":"ContainerDied","Data":"80dfff63d9e4a012a720f0098428629ec825a8f6ffc2602c215aeed0b85fbcf0"} Jun 13 00:56:30.849932 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:30.849850 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:30.850080 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:30.849960 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:30.850080 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:30.849960 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:30.850080 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:30.850070 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:30.850243 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:30.850089 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:30.850243 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:30.850171 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:32.007328 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:32.007294 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:32.008065 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:32.007476 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 13 00:56:32.008065 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:32.007925 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wntlh" Jun 13 00:56:32.849653 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:32.849576 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:32.849653 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:32.849613 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:32.849924 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:32.849576 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:32.849924 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:32.849697 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:56:32.849924 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:32.849756 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl9fb" podUID="494eb8c5-81a4-43c5-9f45-64944e129a60" Jun 13 00:56:32.849924 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:32.849888 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5sblb" podUID="ae75009f-6dcc-49b0-8d6a-56b6e2813f00" Jun 13 00:56:34.272392 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.272372 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-107.ec2.internal" event="NodeReady" Jun 13 00:56:34.272731 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.272465 2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 13 00:56:34.313537 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.313506 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-tq4xr"] Jun 13 00:56:34.327242 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.327223 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-869f996f96-dnd7g"] Jun 13 00:56:34.327397 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.327376 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:34.330169 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.330149 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 13 00:56:34.330421 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.330408 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8gq9q\"" Jun 13 00:56:34.339713 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.339695 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 13 00:56:34.342560 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.342542 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-tq4xr"] Jun 13 00:56:34.342638 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.342617 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.344319 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.344301 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f5lsk"] Jun 13 00:56:34.346799 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.346295 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 13 00:56:34.346903 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.346811 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 13 00:56:34.347984 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.347138 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 13 00:56:34.355410 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.355394 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wxfsc\"" Jun 13 00:56:34.362996 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.362975 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w4kkf"] Jun 13 00:56:34.363243 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.363224 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.364612 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.364591 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 13 00:56:34.377142 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.377119 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-869f996f96-dnd7g"] Jun 13 00:56:34.377247 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.377151 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w4kkf"] Jun 13 00:56:34.377247 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.377218 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 13 00:56:34.377247 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.377218 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w4kkf" Jun 13 00:56:34.377592 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.377574 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 13 00:56:34.377674 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.377661 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7d8x8\"" Jun 13 00:56:34.380200 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.380185 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bnjss\"" Jun 13 00:56:34.387413 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.387395 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 13 00:56:34.387413 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.387402 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 13 00:56:34.387895 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.387867 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 13 00:56:34.388769 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.388752 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f5lsk"] Jun 13 00:56:34.428228 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428205 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-installation-pull-secrets\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.428330 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428238 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-trusted-ca\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.428330 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428278 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfa7355d-fa80-4074-8574-844ade0436f7-ca-trust-extracted\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.428330 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428311 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-bound-sa-token\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.428430 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428397 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-registry-certificates\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.428430 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428419 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7140ef67-07de-4771-b594-0a3c63f7254c-nginx-conf\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:34.428500 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428433 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2rh5\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-kube-api-access-d2rh5\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.428500 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428451 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-image-registry-private-configuration\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.428500 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428470 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:34.428500 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.428493 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.529759 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.529698 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-installation-pull-secrets\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.529759 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.529736 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-trusted-ca\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.529960 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.529764 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:34.529960 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.529818 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz789\" (UniqueName: \"kubernetes.io/projected/fa4f439a-83ef-4ba0-a9c6-ae5d1655d033-kube-api-access-jz789\") pod \"ingress-canary-w4kkf\" (UID: \"fa4f439a-83ef-4ba0-a9c6-ae5d1655d033\") " pod="openshift-ingress-canary/ingress-canary-w4kkf" Jun 13 00:56:34.529960 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.529856 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfa7355d-fa80-4074-8574-844ade0436f7-ca-trust-extracted\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.529960 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.529944 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:34.530150 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.529944 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-bound-sa-token\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.530150 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.530005 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 00:57:06.529986139 +0000 UTC m=+66.103915006 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 13 00:56:34.530150 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530034 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.530150 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530066 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:34.530150 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.530124 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:34.530358 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530123 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-registry-certificates\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.530358 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.530162 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret podName:ae75009f-6dcc-49b0-8d6a-56b6e2813f00 nodeName:}" failed. No retries permitted until 2026-06-13 00:57:06.530148635 +0000 UTC m=+66.104077500 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret") pod "global-pull-secret-syncer-5sblb" (UID: "ae75009f-6dcc-49b0-8d6a-56b6e2813f00") : object "kube-system"/"original-pull-secret" not registered Jun 13 00:56:34.530358 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530194 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d35bfb84-8978-47a6-8c05-034fb69773dd-config-volume\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.530358 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530212 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfa7355d-fa80-4074-8574-844ade0436f7-ca-trust-extracted\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.530358 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530215 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d35bfb84-8978-47a6-8c05-034fb69773dd-tmp-dir\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.530509 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530273 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7140ef67-07de-4771-b594-0a3c63f7254c-nginx-conf\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:34.530542 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530517 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2rh5\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-kube-api-access-d2rh5\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.530579 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530546 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8jq2\" (UniqueName: \"kubernetes.io/projected/d35bfb84-8978-47a6-8c05-034fb69773dd-kube-api-access-p8jq2\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.530627 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530578 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-image-registry-private-configuration\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.530627 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530610 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:34.530723 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530632 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-registry-certificates\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.530723 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530681 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-trusted-ca\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.530723 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530636 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.530723 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.530722 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 13 00:56:34.530930 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530730 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa4f439a-83ef-4ba0-a9c6-ae5d1655d033-cert\") pod \"ingress-canary-w4kkf\" (UID: \"fa4f439a-83ef-4ba0-a9c6-ae5d1655d033\") " pod="openshift-ingress-canary/ingress-canary-w4kkf" Jun 13 00:56:34.530930 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.530735 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-869f996f96-dnd7g: secret "image-registry-tls" not found Jun 13 00:56:34.530930 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.530831 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 13 00:56:34.530930 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.530855 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls podName:dfa7355d-fa80-4074-8574-844ade0436f7 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:35.030836318 +0000 UTC m=+34.604765185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls") pod "image-registry-869f996f96-dnd7g" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7") : secret "image-registry-tls" not found Jun 13 00:56:34.530930 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.530875 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert podName:7140ef67-07de-4771-b594-0a3c63f7254c nodeName:}" failed. No retries permitted until 2026-06-13 00:56:35.030864615 +0000 UTC m=+34.604793480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-tq4xr" (UID: "7140ef67-07de-4771-b594-0a3c63f7254c") : secret "networking-console-plugin-cert" not found Jun 13 00:56:34.530930 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.530906 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7140ef67-07de-4771-b594-0a3c63f7254c-nginx-conf\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:34.533873 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.533854 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-installation-pull-secrets\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.533954 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.533857 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-image-registry-private-configuration\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.538132 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.538109 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-bound-sa-token\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.539716 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.539691 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2rh5\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-kube-api-access-d2rh5\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:34.632058 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.632035 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa4f439a-83ef-4ba0-a9c6-ae5d1655d033-cert\") pod \"ingress-canary-w4kkf\" (UID: \"fa4f439a-83ef-4ba0-a9c6-ae5d1655d033\") " pod="openshift-ingress-canary/ingress-canary-w4kkf" Jun 13 00:56:34.632199 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.632089 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz789\" (UniqueName: \"kubernetes.io/projected/fa4f439a-83ef-4ba0-a9c6-ae5d1655d033-kube-api-access-jz789\") pod \"ingress-canary-w4kkf\" (UID: \"fa4f439a-83ef-4ba0-a9c6-ae5d1655d033\") " pod="openshift-ingress-canary/ingress-canary-w4kkf" Jun 13 00:56:34.632199 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.632114 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:34.632199 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.632139 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.632199 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.632190 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d35bfb84-8978-47a6-8c05-034fb69773dd-config-volume\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.632390 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.632212 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d35bfb84-8978-47a6-8c05-034fb69773dd-tmp-dir\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.632390 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.632225 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 13 00:56:34.632390 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.632240 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8jq2\" (UniqueName: \"kubernetes.io/projected/d35bfb84-8978-47a6-8c05-034fb69773dd-kube-api-access-p8jq2\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.632390 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.632248 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 13 00:56:34.632390 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.632261 2552 projected.go:196] Error preparing data for projected volume kube-api-access-rfxkx for pod openshift-network-diagnostics/network-check-target-nl9fb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:34.632390 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.632296 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 13 00:56:34.632390 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.632306 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx podName:494eb8c5-81a4-43c5-9f45-64944e129a60 nodeName:}" failed. No retries permitted until 2026-06-13 00:57:06.632291969 +0000 UTC m=+66.206220837 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfxkx" (UniqueName: "kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx") pod "network-check-target-nl9fb" (UID: "494eb8c5-81a4-43c5-9f45-64944e129a60") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 13 00:56:34.632390 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:34.632361 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls podName:d35bfb84-8978-47a6-8c05-034fb69773dd nodeName:}" failed. No retries permitted until 2026-06-13 00:56:35.132344183 +0000 UTC m=+34.706273063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls") pod "dns-default-f5lsk" (UID: "d35bfb84-8978-47a6-8c05-034fb69773dd") : secret "dns-default-metrics-tls" not found Jun 13 00:56:34.632668 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.632571 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d35bfb84-8978-47a6-8c05-034fb69773dd-tmp-dir\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.632753 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.632734 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d35bfb84-8978-47a6-8c05-034fb69773dd-config-volume\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.634235 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.634218 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa4f439a-83ef-4ba0-a9c6-ae5d1655d033-cert\") pod \"ingress-canary-w4kkf\" (UID: \"fa4f439a-83ef-4ba0-a9c6-ae5d1655d033\") " pod="openshift-ingress-canary/ingress-canary-w4kkf" Jun 13 00:56:34.640075 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.640053 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8jq2\" (UniqueName: \"kubernetes.io/projected/d35bfb84-8978-47a6-8c05-034fb69773dd-kube-api-access-p8jq2\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:34.640688 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.640673 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz789\" (UniqueName: \"kubernetes.io/projected/fa4f439a-83ef-4ba0-a9c6-ae5d1655d033-kube-api-access-jz789\") pod \"ingress-canary-w4kkf\" (UID: \"fa4f439a-83ef-4ba0-a9c6-ae5d1655d033\") " pod="openshift-ingress-canary/ingress-canary-w4kkf" Jun 13 00:56:34.687067 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.687037 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w4kkf" Jun 13 00:56:34.849272 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.849240 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:56:34.849272 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.849259 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:56:34.849461 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.849280 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:56:34.851911 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.851893 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 13 00:56:34.852036 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.851931 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 13 00:56:34.852036 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.852000 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 13 00:56:34.852146 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.852046 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 13 00:56:34.852629 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.852611 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m8nk7\"" Jun 13 00:56:34.852629 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.852615 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-clqbp\"" Jun 13 00:56:34.878024 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.878006 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w4kkf"] Jun 13 00:56:34.881649 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:56:34.881630 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa4f439a_83ef_4ba0_a9c6_ae5d1655d033.slice/crio-2ca0c23e3fb29dcb822d0be951499a33f2849c8b3b14bc408cba31896c7a1f54 WatchSource:0}: Error finding container 2ca0c23e3fb29dcb822d0be951499a33f2849c8b3b14bc408cba31896c7a1f54: Status 404 returned error can't find the container with id 2ca0c23e3fb29dcb822d0be951499a33f2849c8b3b14bc408cba31896c7a1f54 Jun 13 00:56:34.999453 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:34.999420 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w4kkf" event={"ID":"fa4f439a-83ef-4ba0-a9c6-ae5d1655d033","Type":"ContainerStarted","Data":"2ca0c23e3fb29dcb822d0be951499a33f2849c8b3b14bc408cba31896c7a1f54"} Jun 13 00:56:35.002236 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:35.002211 2552 generic.go:356] "Generic (PLEG): container finished" podID="638de231-57e4-4732-bc73-794b751a3249" containerID="65461c2f62ab60edddf9ddaeff2d9f84da961959fffdc84a7d02bd3fa8482050" exitCode=0 Jun 13 00:56:35.002347 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:35.002254 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" event={"ID":"638de231-57e4-4732-bc73-794b751a3249","Type":"ContainerDied","Data":"65461c2f62ab60edddf9ddaeff2d9f84da961959fffdc84a7d02bd3fa8482050"} Jun 13 00:56:35.036692 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:35.036663 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:35.036772 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:35.036708 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:35.036852 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:35.036835 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 13 00:56:35.036852 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:35.036847 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-869f996f96-dnd7g: secret "image-registry-tls" not found Jun 13 00:56:35.036946 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:35.036890 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls podName:dfa7355d-fa80-4074-8574-844ade0436f7 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:36.03687338 +0000 UTC m=+35.610802262 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls") pod "image-registry-869f996f96-dnd7g" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7") : secret "image-registry-tls" not found Jun 13 00:56:35.036946 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:35.036893 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 13 00:56:35.037023 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:35.036951 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert podName:7140ef67-07de-4771-b594-0a3c63f7254c nodeName:}" failed. No retries permitted until 2026-06-13 00:56:36.036934182 +0000 UTC m=+35.610863062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-tq4xr" (UID: "7140ef67-07de-4771-b594-0a3c63f7254c") : secret "networking-console-plugin-cert" not found Jun 13 00:56:35.137309 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:35.137278 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:35.137430 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:35.137412 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 13 00:56:35.137493 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:35.137471 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls podName:d35bfb84-8978-47a6-8c05-034fb69773dd nodeName:}" failed. No retries permitted until 2026-06-13 00:56:36.137452922 +0000 UTC m=+35.711381791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls") pod "dns-default-f5lsk" (UID: "d35bfb84-8978-47a6-8c05-034fb69773dd") : secret "dns-default-metrics-tls" not found Jun 13 00:56:36.008002 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:36.007968 2552 generic.go:356] "Generic (PLEG): container finished" podID="638de231-57e4-4732-bc73-794b751a3249" containerID="94685e7400703af68032f39221a2dcbf6f65afba49254639007fa0057a24d204" exitCode=0 Jun 13 00:56:36.008505 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:36.008022 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" event={"ID":"638de231-57e4-4732-bc73-794b751a3249","Type":"ContainerDied","Data":"94685e7400703af68032f39221a2dcbf6f65afba49254639007fa0057a24d204"} Jun 13 00:56:36.045237 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:36.045207 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:36.045390 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:36.045242 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:36.045390 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:36.045353 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 13 00:56:36.045489 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:36.045411 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert podName:7140ef67-07de-4771-b594-0a3c63f7254c nodeName:}" failed. No retries permitted until 2026-06-13 00:56:38.045395591 +0000 UTC m=+37.619324461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-tq4xr" (UID: "7140ef67-07de-4771-b594-0a3c63f7254c") : secret "networking-console-plugin-cert" not found Jun 13 00:56:36.045489 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:36.045356 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 13 00:56:36.045489 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:36.045454 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-869f996f96-dnd7g: secret "image-registry-tls" not found Jun 13 00:56:36.045618 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:36.045493 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls podName:dfa7355d-fa80-4074-8574-844ade0436f7 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:38.045482152 +0000 UTC m=+37.619411020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls") pod "image-registry-869f996f96-dnd7g" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7") : secret "image-registry-tls" not found Jun 13 00:56:36.146334 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:36.146298 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:36.146509 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:36.146452 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 13 00:56:36.146587 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:36.146529 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls podName:d35bfb84-8978-47a6-8c05-034fb69773dd nodeName:}" failed. No retries permitted until 2026-06-13 00:56:38.146507638 +0000 UTC m=+37.720436521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls") pod "dns-default-f5lsk" (UID: "d35bfb84-8978-47a6-8c05-034fb69773dd") : secret "dns-default-metrics-tls" not found Jun 13 00:56:37.012437 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:37.012218 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w4kkf" event={"ID":"fa4f439a-83ef-4ba0-a9c6-ae5d1655d033","Type":"ContainerStarted","Data":"14202f32f8b4a01872aa63b93da13e41bb7dfaed25415edd7090f1a7960a404f"} Jun 13 00:56:37.015610 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:37.015581 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" event={"ID":"638de231-57e4-4732-bc73-794b751a3249","Type":"ContainerStarted","Data":"8b305ed5b50e1d3fd8fda98f78cec7245a8cb89dd53f6ba386b42c370bd144f5"} Jun 13 00:56:37.033597 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:37.033548 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w4kkf" podStartSLOduration=1.092368821 podStartE2EDuration="3.033533124s" totalImagesPullingTime="1.941164303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:37.032812241 +0000 UTC m=+36.606741149" watchObservedRunningTime="2026-06-13 00:56:37.033533124 +0000 UTC m=+36.607462032" Jun 13 00:56:37.059656 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:37.059611 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tqx2r" podStartSLOduration=8.956177312 podStartE2EDuration="36.059593439s" totalImagesPullingTime="27.103416127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:56:37.058666066 +0000 UTC m=+36.632594980" watchObservedRunningTime="2026-06-13 00:56:37.059593439 +0000 UTC m=+36.633522348" Jun 13 00:56:38.061346 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:38.061313 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:38.061346 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:38.061347 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:38.061767 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:38.061453 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 13 00:56:38.061767 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:38.061469 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 13 00:56:38.061767 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:38.061479 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-869f996f96-dnd7g: secret "image-registry-tls" not found Jun 13 00:56:38.061767 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:38.061518 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert podName:7140ef67-07de-4771-b594-0a3c63f7254c nodeName:}" failed. No retries permitted until 2026-06-13 00:56:42.061499876 +0000 UTC m=+41.635428757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-tq4xr" (UID: "7140ef67-07de-4771-b594-0a3c63f7254c") : secret "networking-console-plugin-cert" not found Jun 13 00:56:38.061767 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:38.061530 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls podName:dfa7355d-fa80-4074-8574-844ade0436f7 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:42.061524858 +0000 UTC m=+41.635453723 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls") pod "image-registry-869f996f96-dnd7g" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7") : secret "image-registry-tls" not found Jun 13 00:56:38.162495 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:38.162463 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:38.162650 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:38.162611 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 13 00:56:38.162718 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:38.162672 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls podName:d35bfb84-8978-47a6-8c05-034fb69773dd nodeName:}" failed. No retries permitted until 2026-06-13 00:56:42.162653402 +0000 UTC m=+41.736582269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls") pod "dns-default-f5lsk" (UID: "d35bfb84-8978-47a6-8c05-034fb69773dd") : secret "dns-default-metrics-tls" not found Jun 13 00:56:42.085922 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:42.085886 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:42.085922 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:42.085926 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:42.086420 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:42.086018 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 13 00:56:42.086420 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:42.086019 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 13 00:56:42.086420 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:42.086028 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-869f996f96-dnd7g: secret "image-registry-tls" not found Jun 13 00:56:42.086420 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:42.086076 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert podName:7140ef67-07de-4771-b594-0a3c63f7254c nodeName:}" failed. No retries permitted until 2026-06-13 00:56:50.086061198 +0000 UTC m=+49.659990066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-tq4xr" (UID: "7140ef67-07de-4771-b594-0a3c63f7254c") : secret "networking-console-plugin-cert" not found Jun 13 00:56:42.086420 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:42.086114 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls podName:dfa7355d-fa80-4074-8574-844ade0436f7 nodeName:}" failed. No retries permitted until 2026-06-13 00:56:50.086098903 +0000 UTC m=+49.660027769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls") pod "image-registry-869f996f96-dnd7g" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7") : secret "image-registry-tls" not found Jun 13 00:56:42.186341 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:42.186311 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:42.186481 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:42.186440 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 13 00:56:42.186528 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:42.186489 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls podName:d35bfb84-8978-47a6-8c05-034fb69773dd nodeName:}" failed. No retries permitted until 2026-06-13 00:56:50.186477375 +0000 UTC m=+49.760406240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls") pod "dns-default-f5lsk" (UID: "d35bfb84-8978-47a6-8c05-034fb69773dd") : secret "dns-default-metrics-tls" not found Jun 13 00:56:50.140936 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:50.140896 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:56:50.140936 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:50.140939 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:56:50.141417 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:50.141035 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 13 00:56:50.141417 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:50.141047 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-869f996f96-dnd7g: secret "image-registry-tls" not found Jun 13 00:56:50.141417 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:50.141062 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 13 00:56:50.141417 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:50.141094 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls podName:dfa7355d-fa80-4074-8574-844ade0436f7 nodeName:}" failed. No retries permitted until 2026-06-13 00:57:06.141080947 +0000 UTC m=+65.715009813 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls") pod "image-registry-869f996f96-dnd7g" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7") : secret "image-registry-tls" not found Jun 13 00:56:50.141417 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:50.141114 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert podName:7140ef67-07de-4771-b594-0a3c63f7254c nodeName:}" failed. No retries permitted until 2026-06-13 00:57:06.141100681 +0000 UTC m=+65.715029547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-tq4xr" (UID: "7140ef67-07de-4771-b594-0a3c63f7254c") : secret "networking-console-plugin-cert" not found Jun 13 00:56:50.241599 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:56:50.241565 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:56:50.241757 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:50.241702 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 13 00:56:50.241757 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:56:50.241750 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls podName:d35bfb84-8978-47a6-8c05-034fb69773dd nodeName:}" failed. No retries permitted until 2026-06-13 00:57:06.24173754 +0000 UTC m=+65.815666407 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls") pod "dns-default-f5lsk" (UID: "d35bfb84-8978-47a6-8c05-034fb69773dd") : secret "dns-default-metrics-tls" not found Jun 13 00:57:00.996876 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:00.996852 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2998n" Jun 13 00:57:06.155724 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.155689 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:57:06.155724 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.155729 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:57:06.156158 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:06.155834 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 13 00:57:06.156158 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:06.155851 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-869f996f96-dnd7g: secret "image-registry-tls" not found Jun 13 00:57:06.156158 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:06.155833 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 13 00:57:06.156158 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:06.155928 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls podName:dfa7355d-fa80-4074-8574-844ade0436f7 nodeName:}" failed. No retries permitted until 2026-06-13 00:57:38.155912901 +0000 UTC m=+97.729841767 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls") pod "image-registry-869f996f96-dnd7g" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7") : secret "image-registry-tls" not found Jun 13 00:57:06.156158 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:06.155973 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert podName:7140ef67-07de-4771-b594-0a3c63f7254c nodeName:}" failed. No retries permitted until 2026-06-13 00:57:38.155959319 +0000 UTC m=+97.729888184 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-tq4xr" (UID: "7140ef67-07de-4771-b594-0a3c63f7254c") : secret "networking-console-plugin-cert" not found Jun 13 00:57:06.256886 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.256860 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:57:06.257018 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:06.256988 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 13 00:57:06.257068 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:06.257036 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls podName:d35bfb84-8978-47a6-8c05-034fb69773dd nodeName:}" failed. No retries permitted until 2026-06-13 00:57:38.257023474 +0000 UTC m=+97.830952341 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls") pod "dns-default-f5lsk" (UID: "d35bfb84-8978-47a6-8c05-034fb69773dd") : secret "dns-default-metrics-tls" not found Jun 13 00:57:06.558267 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.558243 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:57:06.558395 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.558306 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:57:06.560955 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.560938 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 13 00:57:06.561033 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.560941 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 13 00:57:06.568996 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:06.568979 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 13 00:57:06.569090 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:06.569044 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 00:58:10.569028224 +0000 UTC m=+130.142957105 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : secret "metrics-daemon-secret" not found Jun 13 00:57:06.572057 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.572041 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae75009f-6dcc-49b0-8d6a-56b6e2813f00-original-pull-secret\") pod \"global-pull-secret-syncer-5sblb\" (UID: \"ae75009f-6dcc-49b0-8d6a-56b6e2813f00\") " pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:57:06.659001 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.658978 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5sblb" Jun 13 00:57:06.659156 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.659137 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:57:06.661810 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.661795 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 13 00:57:06.672245 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.672230 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 13 00:57:06.682288 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.682255 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxkx\" (UniqueName: \"kubernetes.io/projected/494eb8c5-81a4-43c5-9f45-64944e129a60-kube-api-access-rfxkx\") pod \"network-check-target-nl9fb\" (UID: \"494eb8c5-81a4-43c5-9f45-64944e129a60\") " pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:57:06.814909 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.814846 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5sblb"] Jun 13 00:57:06.818169 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:57:06.818147 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae75009f_6dcc_49b0_8d6a_56b6e2813f00.slice/crio-82c71a71a8f0e77104fb8bc64c6e7b9f8e02b58d13231ffd33a08ff6a7f5a0b1 WatchSource:0}: Error finding container 82c71a71a8f0e77104fb8bc64c6e7b9f8e02b58d13231ffd33a08ff6a7f5a0b1: Status 404 returned error can't find the container with id 82c71a71a8f0e77104fb8bc64c6e7b9f8e02b58d13231ffd33a08ff6a7f5a0b1 Jun 13 00:57:06.972514 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.972484 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-clqbp\"" Jun 13 00:57:06.980190 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:06.980172 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:57:07.085983 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:07.085952 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5sblb" event={"ID":"ae75009f-6dcc-49b0-8d6a-56b6e2813f00","Type":"ContainerStarted","Data":"82c71a71a8f0e77104fb8bc64c6e7b9f8e02b58d13231ffd33a08ff6a7f5a0b1"} Jun 13 00:57:07.117621 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:07.117594 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nl9fb"] Jun 13 00:57:07.120537 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:57:07.120515 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494eb8c5_81a4_43c5_9f45_64944e129a60.slice/crio-1b494ebe14194d650ac44bfd20cbdba18ab48d9e98c96e3d8f51cf22f52921e7 WatchSource:0}: Error finding container 1b494ebe14194d650ac44bfd20cbdba18ab48d9e98c96e3d8f51cf22f52921e7: Status 404 returned error can't find the container with id 1b494ebe14194d650ac44bfd20cbdba18ab48d9e98c96e3d8f51cf22f52921e7 Jun 13 00:57:08.091151 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:08.091111 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nl9fb" event={"ID":"494eb8c5-81a4-43c5-9f45-64944e129a60","Type":"ContainerStarted","Data":"1b494ebe14194d650ac44bfd20cbdba18ab48d9e98c96e3d8f51cf22f52921e7"} Jun 13 00:57:12.102791 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:12.102746 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5sblb" event={"ID":"ae75009f-6dcc-49b0-8d6a-56b6e2813f00","Type":"ContainerStarted","Data":"f4c114be13e0d5113d246001e8f5b2a950bc27ed48988e725c71bfe017418486"} Jun 13 00:57:12.104237 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:12.104209 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nl9fb" event={"ID":"494eb8c5-81a4-43c5-9f45-64944e129a60","Type":"ContainerStarted","Data":"2c1f427fed1c86278e0a0065cb270e8b3ce63be176746aede9589f8278fc5050"} Jun 13 00:57:12.104352 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:12.104260 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:57:12.119575 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:12.119530 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5sblb" podStartSLOduration=66.598561006 podStartE2EDuration="1m11.119514017s" totalImagesPullingTime="4.520953011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:57:12.118724849 +0000 UTC m=+71.692653736" watchObservedRunningTime="2026-06-13 00:57:12.119514017 +0000 UTC m=+71.693442883" Jun 13 00:57:12.135182 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:12.135143 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nl9fb" podStartSLOduration=66.919455629 podStartE2EDuration="1m11.135130278s" totalImagesPullingTime="4.215674649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:57:12.13444893 +0000 UTC m=+71.708377818" watchObservedRunningTime="2026-06-13 00:57:12.135130278 +0000 UTC m=+71.709059144" Jun 13 00:57:38.184370 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:38.184337 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:57:38.184370 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:38.184371 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:57:38.184896 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:38.184467 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 13 00:57:38.184896 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:38.184478 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-869f996f96-dnd7g: secret "image-registry-tls" not found Jun 13 00:57:38.184896 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:38.184525 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls podName:dfa7355d-fa80-4074-8574-844ade0436f7 nodeName:}" failed. No retries permitted until 2026-06-13 00:58:42.184511474 +0000 UTC m=+161.758440341 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls") pod "image-registry-869f996f96-dnd7g" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7") : secret "image-registry-tls" not found Jun 13 00:57:38.184896 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:38.184466 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 13 00:57:38.184896 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:38.184579 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert podName:7140ef67-07de-4771-b594-0a3c63f7254c nodeName:}" failed. No retries permitted until 2026-06-13 00:58:42.184565577 +0000 UTC m=+161.758494443 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-tq4xr" (UID: "7140ef67-07de-4771-b594-0a3c63f7254c") : secret "networking-console-plugin-cert" not found Jun 13 00:57:38.285182 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:38.285157 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:57:38.285268 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:38.285251 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 13 00:57:38.285310 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:57:38.285297 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls podName:d35bfb84-8978-47a6-8c05-034fb69773dd nodeName:}" failed. No retries permitted until 2026-06-13 00:58:42.285286846 +0000 UTC m=+161.859215711 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls") pod "dns-default-f5lsk" (UID: "d35bfb84-8978-47a6-8c05-034fb69773dd") : secret "dns-default-metrics-tls" not found Jun 13 00:57:43.108814 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:57:43.108765 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nl9fb" Jun 13 00:58:10.609938 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:58:10.609884 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:58:10.610436 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:10.610035 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 13 00:58:10.610436 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:10.610108 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:12.610092295 +0000 UTC m=+252.184021161 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : secret "metrics-daemon-secret" not found Jun 13 00:58:37.338351 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:37.338296 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" podUID="7140ef67-07de-4771-b594-0a3c63f7254c" Jun 13 00:58:37.354455 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:37.354428 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" podUID="dfa7355d-fa80-4074-8574-844ade0436f7" Jun 13 00:58:37.371575 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:37.371550 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-f5lsk" podUID="d35bfb84-8978-47a6-8c05-034fb69773dd" Jun 13 00:58:37.865830 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:37.865768 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 00:58:38.300134 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:58:38.300105 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:58:38.300134 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:58:38.300130 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f5lsk" Jun 13 00:58:38.300300 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:58:38.300105 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:58:42.211385 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:58:42.211290 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 00:58:42.211385 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:58:42.211377 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 00:58:42.211906 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:42.211445 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 13 00:58:42.211906 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:42.211470 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-869f996f96-dnd7g: secret "image-registry-tls" not found Jun 13 00:58:42.211906 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:42.211453 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 13 00:58:42.211906 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:42.211532 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls podName:dfa7355d-fa80-4074-8574-844ade0436f7 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:44.211514873 +0000 UTC m=+283.785443739 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls") pod "image-registry-869f996f96-dnd7g" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7") : secret "image-registry-tls" not found Jun 13 00:58:42.211906 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:42.211546 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert podName:7140ef67-07de-4771-b594-0a3c63f7254c nodeName:}" failed. No retries permitted until 2026-06-13 01:00:44.211539169 +0000 UTC m=+283.785468034 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-tq4xr" (UID: "7140ef67-07de-4771-b594-0a3c63f7254c") : secret "networking-console-plugin-cert" not found Jun 13 00:58:42.311753 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:58:42.311719 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 00:58:42.311900 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:42.311875 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 13 00:58:42.311938 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:58:42.311932 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls podName:d35bfb84-8978-47a6-8c05-034fb69773dd nodeName:}" failed. No retries permitted until 2026-06-13 01:00:44.311916188 +0000 UTC m=+283.885845074 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls") pod "dns-default-f5lsk" (UID: "d35bfb84-8978-47a6-8c05-034fb69773dd") : secret "dns-default-metrics-tls" not found Jun 13 00:58:51.849421 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:58:51.849382 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 00:59:58.446399 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.446368 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp"] Jun 13 00:59:58.448224 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.448205 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.455347 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.455324 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck"] Jun 13 00:59:58.457023 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.456995 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5"] Jun 13 00:59:58.457116 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.457098 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:58.458688 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.458670 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 00:59:58.460081 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.460066 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 13 00:59:58.460168 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.460117 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 13 00:59:58.460230 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.460177 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 13 00:59:58.460910 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.460895 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 13 00:59:58.460983 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.460966 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-29g65\"" Jun 13 00:59:58.461021 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.460994 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-k4wtk\"" Jun 13 00:59:58.462501 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.462484 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 13 00:59:58.462594 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.462506 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 13 00:59:58.462594 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.462538 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 13 00:59:58.462594 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.462550 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 13 00:59:58.462594 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.462561 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 13 00:59:58.462875 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.462859 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 13 00:59:58.462935 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.462860 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 13 00:59:58.462935 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.462899 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kg2ql\"" Jun 13 00:59:58.467944 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.467923 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp"] Jun 13 00:59:58.480863 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.480843 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck"] Jun 13 00:59:58.482208 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.482183 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5"] Jun 13 00:59:58.517319 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.517296 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7edddc5-6c88-4ba0-9165-9bb0e62220a1-serving-cert\") pod \"service-ca-operator-7ccbfd59dc-q9jvp\" (UID: \"f7edddc5-6c88-4ba0-9165-9bb0e62220a1\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.517445 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.517321 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/210bbc79-fa75-47da-9a3e-69b702fa8cff-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:58.517445 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.517340 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:58.517445 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.517357 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q8kz\" (UniqueName: \"kubernetes.io/projected/21318529-f4f2-46cf-808e-4a5786203157-kube-api-access-8q8kz\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 00:59:58.517445 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.517397 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 00:59:58.517445 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.517414 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7edddc5-6c88-4ba0-9165-9bb0e62220a1-config\") pod \"service-ca-operator-7ccbfd59dc-q9jvp\" (UID: \"f7edddc5-6c88-4ba0-9165-9bb0e62220a1\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.517445 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.517446 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjblr\" (UniqueName: \"kubernetes.io/projected/f7edddc5-6c88-4ba0-9165-9bb0e62220a1-kube-api-access-cjblr\") pod \"service-ca-operator-7ccbfd59dc-q9jvp\" (UID: \"f7edddc5-6c88-4ba0-9165-9bb0e62220a1\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.517636 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.517503 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s5g2\" (UniqueName: \"kubernetes.io/projected/210bbc79-fa75-47da-9a3e-69b702fa8cff-kube-api-access-9s5g2\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:58.544763 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.544744 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z"] Jun 13 00:59:58.546406 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.546390 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z" Jun 13 00:59:58.552981 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.552966 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-f6kr8\"" Jun 13 00:59:58.553534 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.553520 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 13 00:59:58.553824 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.553807 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 13 00:59:58.561167 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.561147 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn"] Jun 13 00:59:58.563083 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.563070 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-797xq"] Jun 13 00:59:58.563198 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.563184 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.565249 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.565222 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6947749cfc-797xq" Jun 13 00:59:58.565443 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.565424 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 13 00:59:58.565867 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.565854 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-7f6mh"] Jun 13 00:59:58.569688 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.569668 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 13 00:59:58.569853 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.569828 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-bdjw2\"" Jun 13 00:59:58.570020 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.569816 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.570097 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.570034 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gzd99\"" Jun 13 00:59:58.570309 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.570282 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 13 00:59:58.570309 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.569729 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-788bb8b696-bhgjk"] Jun 13 00:59:58.570697 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.570673 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 13 00:59:58.573530 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.573511 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z"] Jun 13 00:59:58.573701 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.573514 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 13 00:59:58.573918 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.573580 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.575286 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.575106 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 13 00:59:58.575286 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.575258 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 13 00:59:58.575610 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.575565 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-spjhw\"" Jun 13 00:59:58.575610 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.575596 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 13 00:59:58.579627 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.579453 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 13 00:59:58.579627 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.579456 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 13 00:59:58.579627 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.579535 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 13 00:59:58.579627 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.579546 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 13 00:59:58.580051 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.579669 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 13 00:59:58.580051 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.579968 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 13 00:59:58.580355 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.580131 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-r6z2s\"" Jun 13 00:59:58.583288 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.583270 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 13 00:59:58.587399 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.587380 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-797xq"] Jun 13 00:59:58.589314 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.589293 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-7f6mh"] Jun 13 00:59:58.590409 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.590389 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn"] Jun 13 00:59:58.593095 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.593074 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-788bb8b696-bhgjk"] Jun 13 00:59:58.618041 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618023 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s5g2\" (UniqueName: \"kubernetes.io/projected/210bbc79-fa75-47da-9a3e-69b702fa8cff-kube-api-access-9s5g2\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:58.618129 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618051 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e85773c6-ab14-45ea-8df1-20975bf51131-config\") pod \"kube-storage-version-migrator-operator-7c578b78fc-lchjn\" (UID: \"e85773c6-ab14-45ea-8df1-20975bf51131\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.618129 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618074 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/210bbc79-fa75-47da-9a3e-69b702fa8cff-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:58.618129 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618089 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q8kz\" (UniqueName: \"kubernetes.io/projected/21318529-f4f2-46cf-808e-4a5786203157-kube-api-access-8q8kz\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 00:59:58.618277 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618156 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp2wz\" (UniqueName: \"kubernetes.io/projected/e85773c6-ab14-45ea-8df1-20975bf51131-kube-api-access-zp2wz\") pod \"kube-storage-version-migrator-operator-7c578b78fc-lchjn\" (UID: \"e85773c6-ab14-45ea-8df1-20975bf51131\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.618277 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618195 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p58k4\" (UniqueName: \"kubernetes.io/projected/cd6b4bd6-7a65-4a18-ae52-a2e8dab2abce-kube-api-access-p58k4\") pod \"network-check-source-6947749cfc-797xq\" (UID: \"cd6b4bd6-7a65-4a18-ae52-a2e8dab2abce\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-797xq" Jun 13 00:59:58.618391 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618272 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7edddc5-6c88-4ba0-9165-9bb0e62220a1-serving-cert\") pod \"service-ca-operator-7ccbfd59dc-q9jvp\" (UID: \"f7edddc5-6c88-4ba0-9165-9bb0e62220a1\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.618391 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618312 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:58.618391 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618363 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 00:59:58.618391 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618389 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7edddc5-6c88-4ba0-9165-9bb0e62220a1-config\") pod \"service-ca-operator-7ccbfd59dc-q9jvp\" (UID: \"f7edddc5-6c88-4ba0-9165-9bb0e62220a1\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.618595 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618418 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttkjx\" (UniqueName: \"kubernetes.io/projected/1622fbd1-cbee-4f68-a7ae-8a7b410dd1e6-kube-api-access-ttkjx\") pod \"volume-data-source-validator-bd4ccd799-qjn9z\" (UID: \"1622fbd1-cbee-4f68-a7ae-8a7b410dd1e6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z" Jun 13 00:59:58.618595 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:58.618443 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 13 00:59:58.618595 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:58.618454 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 13 00:59:58.618595 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618469 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e85773c6-ab14-45ea-8df1-20975bf51131-serving-cert\") pod \"kube-storage-version-migrator-operator-7c578b78fc-lchjn\" (UID: \"e85773c6-ab14-45ea-8df1-20975bf51131\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.618595 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:58.618504 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls podName:210bbc79-fa75-47da-9a3e-69b702fa8cff nodeName:}" failed. No retries permitted until 2026-06-13 00:59:59.118486892 +0000 UTC m=+238.692415768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-2cjck" (UID: "210bbc79-fa75-47da-9a3e-69b702fa8cff") : secret "cluster-monitoring-operator-tls" not found Jun 13 00:59:58.618595 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618522 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjblr\" (UniqueName: \"kubernetes.io/projected/f7edddc5-6c88-4ba0-9165-9bb0e62220a1-kube-api-access-cjblr\") pod \"service-ca-operator-7ccbfd59dc-q9jvp\" (UID: \"f7edddc5-6c88-4ba0-9165-9bb0e62220a1\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.618595 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:58.618582 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls podName:21318529-f4f2-46cf-808e-4a5786203157 nodeName:}" failed. No retries permitted until 2026-06-13 00:59:59.118570564 +0000 UTC m=+238.692499434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls") pod "cluster-samples-operator-9547488fd-gf8p5" (UID: "21318529-f4f2-46cf-808e-4a5786203157") : secret "samples-operator-tls" not found Jun 13 00:59:58.618997 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618878 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7edddc5-6c88-4ba0-9165-9bb0e62220a1-config\") pod \"service-ca-operator-7ccbfd59dc-q9jvp\" (UID: \"f7edddc5-6c88-4ba0-9165-9bb0e62220a1\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.618997 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.618961 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/210bbc79-fa75-47da-9a3e-69b702fa8cff-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:58.620346 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.620328 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7edddc5-6c88-4ba0-9165-9bb0e62220a1-serving-cert\") pod \"service-ca-operator-7ccbfd59dc-q9jvp\" (UID: \"f7edddc5-6c88-4ba0-9165-9bb0e62220a1\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.645768 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.645746 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjblr\" (UniqueName: \"kubernetes.io/projected/f7edddc5-6c88-4ba0-9165-9bb0e62220a1-kube-api-access-cjblr\") pod \"service-ca-operator-7ccbfd59dc-q9jvp\" (UID: \"f7edddc5-6c88-4ba0-9165-9bb0e62220a1\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.655622 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.655600 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q8kz\" (UniqueName: \"kubernetes.io/projected/21318529-f4f2-46cf-808e-4a5786203157-kube-api-access-8q8kz\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 00:59:58.658253 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.658234 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s5g2\" (UniqueName: \"kubernetes.io/projected/210bbc79-fa75-47da-9a3e-69b702fa8cff-kube-api-access-9s5g2\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:58.719609 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719558 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttkjx\" (UniqueName: \"kubernetes.io/projected/1622fbd1-cbee-4f68-a7ae-8a7b410dd1e6-kube-api-access-ttkjx\") pod \"volume-data-source-validator-bd4ccd799-qjn9z\" (UID: \"1622fbd1-cbee-4f68-a7ae-8a7b410dd1e6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z" Jun 13 00:59:58.719609 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719585 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c53466a-47d5-4367-ba1f-de324cb84360-serving-cert\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.719609 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719602 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27knb\" (UniqueName: \"kubernetes.io/projected/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-kube-api-access-27knb\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.719834 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719631 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e85773c6-ab14-45ea-8df1-20975bf51131-serving-cert\") pod \"kube-storage-version-migrator-operator-7c578b78fc-lchjn\" (UID: \"e85773c6-ab14-45ea-8df1-20975bf51131\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.719834 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719680 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-default-certificate\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.719834 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719730 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-stats-auth\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.719834 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719762 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e85773c6-ab14-45ea-8df1-20975bf51131-config\") pod \"kube-storage-version-migrator-operator-7c578b78fc-lchjn\" (UID: \"e85773c6-ab14-45ea-8df1-20975bf51131\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.719834 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719817 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c53466a-47d5-4367-ba1f-de324cb84360-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.720079 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719858 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp2wz\" (UniqueName: \"kubernetes.io/projected/e85773c6-ab14-45ea-8df1-20975bf51131-kube-api-access-zp2wz\") pod \"kube-storage-version-migrator-operator-7c578b78fc-lchjn\" (UID: \"e85773c6-ab14-45ea-8df1-20975bf51131\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.720079 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719895 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c53466a-47d5-4367-ba1f-de324cb84360-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.720079 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719932 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.720079 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719961 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p58k4\" (UniqueName: \"kubernetes.io/projected/cd6b4bd6-7a65-4a18-ae52-a2e8dab2abce-kube-api-access-p58k4\") pod \"network-check-source-6947749cfc-797xq\" (UID: \"cd6b4bd6-7a65-4a18-ae52-a2e8dab2abce\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-797xq" Jun 13 00:59:58.720079 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.719988 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c53466a-47d5-4367-ba1f-de324cb84360-tmp\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.720079 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.720019 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.720079 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.720044 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghhvf\" (UniqueName: \"kubernetes.io/projected/3c53466a-47d5-4367-ba1f-de324cb84360-kube-api-access-ghhvf\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.720079 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.720078 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3c53466a-47d5-4367-ba1f-de324cb84360-snapshots\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.720328 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.720231 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e85773c6-ab14-45ea-8df1-20975bf51131-config\") pod \"kube-storage-version-migrator-operator-7c578b78fc-lchjn\" (UID: \"e85773c6-ab14-45ea-8df1-20975bf51131\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.721519 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.721500 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e85773c6-ab14-45ea-8df1-20975bf51131-serving-cert\") pod \"kube-storage-version-migrator-operator-7c578b78fc-lchjn\" (UID: \"e85773c6-ab14-45ea-8df1-20975bf51131\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.729317 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.729300 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttkjx\" (UniqueName: \"kubernetes.io/projected/1622fbd1-cbee-4f68-a7ae-8a7b410dd1e6-kube-api-access-ttkjx\") pod \"volume-data-source-validator-bd4ccd799-qjn9z\" (UID: \"1622fbd1-cbee-4f68-a7ae-8a7b410dd1e6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z" Jun 13 00:59:58.732121 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.732103 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp2wz\" (UniqueName: \"kubernetes.io/projected/e85773c6-ab14-45ea-8df1-20975bf51131-kube-api-access-zp2wz\") pod \"kube-storage-version-migrator-operator-7c578b78fc-lchjn\" (UID: \"e85773c6-ab14-45ea-8df1-20975bf51131\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.732250 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.732235 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p58k4\" (UniqueName: \"kubernetes.io/projected/cd6b4bd6-7a65-4a18-ae52-a2e8dab2abce-kube-api-access-p58k4\") pod \"network-check-source-6947749cfc-797xq\" (UID: \"cd6b4bd6-7a65-4a18-ae52-a2e8dab2abce\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-797xq" Jun 13 00:59:58.757187 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.757170 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" Jun 13 00:59:58.820583 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.820556 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c53466a-47d5-4367-ba1f-de324cb84360-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.820693 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.820603 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c53466a-47d5-4367-ba1f-de324cb84360-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.820693 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.820622 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.820693 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.820641 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c53466a-47d5-4367-ba1f-de324cb84360-tmp\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.820693 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.820661 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.820839 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:58.820747 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 13 00:59:58.820839 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.820795 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghhvf\" (UniqueName: \"kubernetes.io/projected/3c53466a-47d5-4367-ba1f-de324cb84360-kube-api-access-ghhvf\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.820903 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.820841 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3c53466a-47d5-4367-ba1f-de324cb84360-snapshots\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.820903 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.820877 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c53466a-47d5-4367-ba1f-de324cb84360-serving-cert\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.820962 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:58.820905 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 00:59:59.320881773 +0000 UTC m=+238.894810656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : configmap references non-existent config key: service-ca.crt Jun 13 00:59:58.820962 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.820954 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27knb\" (UniqueName: \"kubernetes.io/projected/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-kube-api-access-27knb\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.821063 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:58.821001 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 00:59:59.320982042 +0000 UTC m=+238.894910944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : secret "router-metrics-certs-default" not found Jun 13 00:59:58.821063 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.821042 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-default-certificate\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.821163 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.821085 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-stats-auth\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.821240 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.821217 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c53466a-47d5-4367-ba1f-de324cb84360-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.821746 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.821688 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c53466a-47d5-4367-ba1f-de324cb84360-tmp\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.821746 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.821741 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3c53466a-47d5-4367-ba1f-de324cb84360-snapshots\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.822431 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.822414 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c53466a-47d5-4367-ba1f-de324cb84360-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.823568 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.823544 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c53466a-47d5-4367-ba1f-de324cb84360-serving-cert\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.823773 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.823755 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-default-certificate\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.823852 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.823798 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-stats-auth\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.831428 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.830580 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghhvf\" (UniqueName: \"kubernetes.io/projected/3c53466a-47d5-4367-ba1f-de324cb84360-kube-api-access-ghhvf\") pod \"insights-operator-5bbd86d6bd-7f6mh\" (UID: \"3c53466a-47d5-4367-ba1f-de324cb84360\") " pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:58.831428 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.831086 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27knb\" (UniqueName: \"kubernetes.io/projected/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-kube-api-access-27knb\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:58.853611 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.853594 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z" Jun 13 00:59:58.875642 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.875615 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp"] Jun 13 00:59:58.876932 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.876913 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" Jun 13 00:59:58.877997 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:59:58.877976 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7edddc5_6c88_4ba0_9165_9bb0e62220a1.slice/crio-2d7669a288ec37623b6b4f89e21ebb846d04ff7e1a002f30b3d8f1e0b9d90eed WatchSource:0}: Error finding container 2d7669a288ec37623b6b4f89e21ebb846d04ff7e1a002f30b3d8f1e0b9d90eed: Status 404 returned error can't find the container with id 2d7669a288ec37623b6b4f89e21ebb846d04ff7e1a002f30b3d8f1e0b9d90eed Jun 13 00:59:58.883392 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.883374 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6947749cfc-797xq" Jun 13 00:59:58.890157 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:58.890138 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" Jun 13 00:59:59.007250 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:59:59.007214 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1622fbd1_cbee_4f68_a7ae_8a7b410dd1e6.slice/crio-ef3ae94c17c29d888e228e08565a0fb8c0edb9d11358042e39bfebed90bac86a WatchSource:0}: Error finding container ef3ae94c17c29d888e228e08565a0fb8c0edb9d11358042e39bfebed90bac86a: Status 404 returned error can't find the container with id ef3ae94c17c29d888e228e08565a0fb8c0edb9d11358042e39bfebed90bac86a Jun 13 00:59:59.012393 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.012328 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z"] Jun 13 00:59:59.030680 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.030658 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn"] Jun 13 00:59:59.033669 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:59:59.033639 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85773c6_ab14_45ea_8df1_20975bf51131.slice/crio-1a66dd12bd9dd0045e59c1f74683c04ded6de9fe411bf70acb8712e709d459be WatchSource:0}: Error finding container 1a66dd12bd9dd0045e59c1f74683c04ded6de9fe411bf70acb8712e709d459be: Status 404 returned error can't find the container with id 1a66dd12bd9dd0045e59c1f74683c04ded6de9fe411bf70acb8712e709d459be Jun 13 00:59:59.040237 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.040212 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-797xq"] Jun 13 00:59:59.043533 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:59:59.043508 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd6b4bd6_7a65_4a18_ae52_a2e8dab2abce.slice/crio-409ff91ca4f23e9ff8c605de0f850413e14b0086d2b5a01ff1623f28a53bbbb1 WatchSource:0}: Error finding container 409ff91ca4f23e9ff8c605de0f850413e14b0086d2b5a01ff1623f28a53bbbb1: Status 404 returned error can't find the container with id 409ff91ca4f23e9ff8c605de0f850413e14b0086d2b5a01ff1623f28a53bbbb1 Jun 13 00:59:59.064611 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.064594 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-7f6mh"] Jun 13 00:59:59.069700 ip-10-0-137-107 kubenswrapper[2552]: W0613 00:59:59.069674 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c53466a_47d5_4367_ba1f_de324cb84360.slice/crio-ce049cf4e82ad6109a305eb34aee4f4e9cf2ac93ad1eb30e77577bd220d095de WatchSource:0}: Error finding container ce049cf4e82ad6109a305eb34aee4f4e9cf2ac93ad1eb30e77577bd220d095de: Status 404 returned error can't find the container with id ce049cf4e82ad6109a305eb34aee4f4e9cf2ac93ad1eb30e77577bd220d095de Jun 13 00:59:59.123610 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.123584 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 00:59:59.123737 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.123640 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 00:59:59.123811 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:59.123746 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 13 00:59:59.123811 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:59.123748 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 13 00:59:59.123923 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:59.123829 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls podName:210bbc79-fa75-47da-9a3e-69b702fa8cff nodeName:}" failed. No retries permitted until 2026-06-13 01:00:00.123807275 +0000 UTC m=+239.697736154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-2cjck" (UID: "210bbc79-fa75-47da-9a3e-69b702fa8cff") : secret "cluster-monitoring-operator-tls" not found Jun 13 00:59:59.123923 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:59.123854 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls podName:21318529-f4f2-46cf-808e-4a5786203157 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:00.123843009 +0000 UTC m=+239.697771876 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls") pod "cluster-samples-operator-9547488fd-gf8p5" (UID: "21318529-f4f2-46cf-808e-4a5786203157") : secret "samples-operator-tls" not found Jun 13 00:59:59.325893 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.325761 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:59.325893 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.325841 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 00:59:59.326177 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:59.325932 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:00.325911974 +0000 UTC m=+239.899840842 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : configmap references non-existent config key: service-ca.crt Jun 13 00:59:59.326177 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:59.325978 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 13 00:59:59.326177 ip-10-0-137-107 kubenswrapper[2552]: E0613 00:59:59.326019 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:00.326007082 +0000 UTC m=+239.899935959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : secret "router-metrics-certs-default" not found Jun 13 00:59:59.499417 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.499336 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" event={"ID":"3c53466a-47d5-4367-ba1f-de324cb84360","Type":"ContainerStarted","Data":"ce049cf4e82ad6109a305eb34aee4f4e9cf2ac93ad1eb30e77577bd220d095de"} Jun 13 00:59:59.502063 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.501531 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6947749cfc-797xq" event={"ID":"cd6b4bd6-7a65-4a18-ae52-a2e8dab2abce","Type":"ContainerStarted","Data":"802bcbceb1d5a355188b04cdf680f68b4dd0b47ae4ffebc1d6a4b501fccea314"} Jun 13 00:59:59.502063 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.501576 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6947749cfc-797xq" event={"ID":"cd6b4bd6-7a65-4a18-ae52-a2e8dab2abce","Type":"ContainerStarted","Data":"409ff91ca4f23e9ff8c605de0f850413e14b0086d2b5a01ff1623f28a53bbbb1"} Jun 13 00:59:59.505820 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.505763 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" event={"ID":"e85773c6-ab14-45ea-8df1-20975bf51131","Type":"ContainerStarted","Data":"1a66dd12bd9dd0045e59c1f74683c04ded6de9fe411bf70acb8712e709d459be"} Jun 13 00:59:59.507351 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.507327 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" event={"ID":"f7edddc5-6c88-4ba0-9165-9bb0e62220a1","Type":"ContainerStarted","Data":"2d7669a288ec37623b6b4f89e21ebb846d04ff7e1a002f30b3d8f1e0b9d90eed"} Jun 13 00:59:59.508650 ip-10-0-137-107 kubenswrapper[2552]: I0613 00:59:59.508624 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z" event={"ID":"1622fbd1-cbee-4f68-a7ae-8a7b410dd1e6","Type":"ContainerStarted","Data":"ef3ae94c17c29d888e228e08565a0fb8c0edb9d11358042e39bfebed90bac86a"} Jun 13 01:00:00.136462 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:00.136415 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 01:00:00.136651 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:00.136492 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 01:00:00.136651 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:00.136630 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 13 01:00:00.136804 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:00.136695 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls podName:21318529-f4f2-46cf-808e-4a5786203157 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:02.136676283 +0000 UTC m=+241.710605153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls") pod "cluster-samples-operator-9547488fd-gf8p5" (UID: "21318529-f4f2-46cf-808e-4a5786203157") : secret "samples-operator-tls" not found Jun 13 01:00:00.137276 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:00.137148 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 13 01:00:00.137276 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:00.137205 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls podName:210bbc79-fa75-47da-9a3e-69b702fa8cff nodeName:}" failed. No retries permitted until 2026-06-13 01:00:02.137189051 +0000 UTC m=+241.711117930 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-2cjck" (UID: "210bbc79-fa75-47da-9a3e-69b702fa8cff") : secret "cluster-monitoring-operator-tls" not found Jun 13 01:00:00.338133 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:00.338092 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:00.338287 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:00.338162 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:00.338632 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:00.338574 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:02.33855301 +0000 UTC m=+241.912481875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : configmap references non-existent config key: service-ca.crt Jun 13 01:00:00.338951 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:00.338930 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 13 01:00:00.339042 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:00.338987 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:02.338970961 +0000 UTC m=+241.912899856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : secret "router-metrics-certs-default" not found Jun 13 01:00:00.513904 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:00.513820 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z" event={"ID":"1622fbd1-cbee-4f68-a7ae-8a7b410dd1e6","Type":"ContainerStarted","Data":"5b8507960ead0668f3a095168052a635c0eec149cc142cd7cbcf9648bedc3f03"} Jun 13 01:00:00.529365 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:00.529312 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-qjn9z" podStartSLOduration=1.340551857 podStartE2EDuration="2.529296791s" totalImagesPullingTime="1.188744934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:00.528116476 +0000 UTC m=+240.102045365" watchObservedRunningTime="2026-06-13 01:00:00.529296791 +0000 UTC m=+240.103225656" Jun 13 01:00:00.529569 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:00.529542 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6947749cfc-797xq" podStartSLOduration=2.5295340570000002 podStartE2EDuration="2.529534057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:59:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 00:59:59.520261248 +0000 UTC m=+239.094190136" watchObservedRunningTime="2026-06-13 01:00:00.529534057 +0000 UTC m=+240.103462946" Jun 13 01:00:02.154370 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.154271 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 01:00:02.154370 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.154336 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 01:00:02.154827 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:02.154442 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 13 01:00:02.154827 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:02.154514 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 13 01:00:02.154827 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:02.154522 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls podName:210bbc79-fa75-47da-9a3e-69b702fa8cff nodeName:}" failed. No retries permitted until 2026-06-13 01:00:06.154502315 +0000 UTC m=+245.728431188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-2cjck" (UID: "210bbc79-fa75-47da-9a3e-69b702fa8cff") : secret "cluster-monitoring-operator-tls" not found Jun 13 01:00:02.154827 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:02.154589 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls podName:21318529-f4f2-46cf-808e-4a5786203157 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:06.154570056 +0000 UTC m=+245.728498931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls") pod "cluster-samples-operator-9547488fd-gf8p5" (UID: "21318529-f4f2-46cf-808e-4a5786203157") : secret "samples-operator-tls" not found Jun 13 01:00:02.356480 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.356442 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:02.356652 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.356497 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:02.356710 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:02.356648 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:06.35662598 +0000 UTC m=+245.930554861 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : configmap references non-existent config key: service-ca.crt Jun 13 01:00:02.356710 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:02.356689 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 13 01:00:02.356807 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:02.356753 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:06.356736994 +0000 UTC m=+245.930665862 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : secret "router-metrics-certs-default" not found Jun 13 01:00:02.522606 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.522511 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" event={"ID":"e85773c6-ab14-45ea-8df1-20975bf51131","Type":"ContainerStarted","Data":"2ce9805cc3bf3feae5916735ab977d6f197be74008cb5626952ce6fad13a11d1"} Jun 13 01:00:02.524197 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.524164 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" event={"ID":"f7edddc5-6c88-4ba0-9165-9bb0e62220a1","Type":"ContainerStarted","Data":"62b91e8426c0f0a11c09b447f85b8be4f1045a0e3d12cc7d1c256a4b2c0fabbe"} Jun 13 01:00:02.525755 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.525707 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" event={"ID":"3c53466a-47d5-4367-ba1f-de324cb84360","Type":"ContainerStarted","Data":"626b835b575a00fae457595fb3b2498e080cccfea52e2dc0a774cfadbffaff8c"} Jun 13 01:00:02.539522 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.539474 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" podStartSLOduration=1.69785831 podStartE2EDuration="4.539457502s" totalImagesPullingTime="2.841599192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:02.537656084 +0000 UTC m=+242.111584973" watchObservedRunningTime="2026-06-13 01:00:02.539457502 +0000 UTC m=+242.113386396" Jun 13 01:00:02.552441 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.552398 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" podStartSLOduration=1.5594556000000002 podStartE2EDuration="4.552388361s" totalImagesPullingTime="2.992932761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:02.551168836 +0000 UTC m=+242.125097725" watchObservedRunningTime="2026-06-13 01:00:02.552388361 +0000 UTC m=+242.126317250" Jun 13 01:00:02.568646 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:02.568607 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" podStartSLOduration=1.761004299 podStartE2EDuration="4.568594924s" totalImagesPullingTime="2.807590625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:02.567687645 +0000 UTC m=+242.141616561" watchObservedRunningTime="2026-06-13 01:00:02.568594924 +0000 UTC m=+242.142523819" Jun 13 01:00:03.369028 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.368986 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kkm5c"] Jun 13 01:00:03.372220 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.372197 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.375311 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.375285 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 13 01:00:03.375416 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.375321 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7pqf\"" Jun 13 01:00:03.375602 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.375583 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 13 01:00:03.392091 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.392068 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kkm5c"] Jun 13 01:00:03.466505 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.466474 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvf8l\" (UniqueName: \"kubernetes.io/projected/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-api-access-fvf8l\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.466670 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.466529 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.466670 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.466637 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb25f6cb-295d-4c25-8614-f3e2a02d4538-data-volume\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.466772 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.466686 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fb25f6cb-295d-4c25-8614-f3e2a02d4538-crio-socket\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.466772 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.466745 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.567795 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.567759 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.567918 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.567836 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvf8l\" (UniqueName: \"kubernetes.io/projected/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-api-access-fvf8l\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.567918 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.567895 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.568007 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:03.567921 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 13 01:00:03.568007 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.567967 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb25f6cb-295d-4c25-8614-f3e2a02d4538-data-volume\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.568007 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:03.567980 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:04.06796141 +0000 UTC m=+243.641890291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : secret "insights-runtime-extractor-tls" not found Jun 13 01:00:03.568114 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:03.568020 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 13 01:00:03.568114 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.568020 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fb25f6cb-295d-4c25-8614-f3e2a02d4538-crio-socket\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.568114 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.568064 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fb25f6cb-295d-4c25-8614-f3e2a02d4538-crio-socket\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.568114 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:03.568075 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:04.068059696 +0000 UTC m=+243.641988576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : configmap "kube-rbac-proxy" not found Jun 13 01:00:03.568246 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.568226 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb25f6cb-295d-4c25-8614-f3e2a02d4538-data-volume\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:03.577210 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:03.577185 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvf8l\" (UniqueName: \"kubernetes.io/projected/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-api-access-fvf8l\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:04.073299 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:04.073261 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:04.073551 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:04.073351 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:04.073551 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:04.073414 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 13 01:00:04.073551 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:04.073461 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 13 01:00:04.073551 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:04.073480 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:05.073463776 +0000 UTC m=+244.647392655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : secret "insights-runtime-extractor-tls" not found Jun 13 01:00:04.073551 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:04.073514 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:05.073502155 +0000 UTC m=+244.647431025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : configmap "kube-rbac-proxy" not found Jun 13 01:00:05.081613 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.081572 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:05.082040 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.081654 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:05.082040 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:05.081742 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 13 01:00:05.082040 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:05.081775 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 13 01:00:05.082040 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:05.081837 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:07.081820161 +0000 UTC m=+246.655749027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : secret "insights-runtime-extractor-tls" not found Jun 13 01:00:05.082040 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:05.081853 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:07.081846032 +0000 UTC m=+246.655774897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : configmap "kube-rbac-proxy" not found Jun 13 01:00:05.531896 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.531824 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-pllx6"] Jun 13 01:00:05.535255 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.535238 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.537620 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.537585 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ppn2d\"" Jun 13 01:00:05.537620 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.537603 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 13 01:00:05.537620 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.537585 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 13 01:00:05.537850 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.537587 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 13 01:00:05.538450 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.538421 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 13 01:00:05.543717 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.543693 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-pllx6"] Jun 13 01:00:05.559704 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.559678 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xf2jd_87376b5e-f663-4b36-affd-55da108f6af7/dns-node-resolver/0.log" Jun 13 01:00:05.685175 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.685147 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw9h7\" (UniqueName: \"kubernetes.io/projected/da8fbab5-c91d-46aa-88f5-0d7cb8b03924-kube-api-access-tw9h7\") pod \"service-ca-7b994bc59-pllx6\" (UID: \"da8fbab5-c91d-46aa-88f5-0d7cb8b03924\") " pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.685315 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.685210 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da8fbab5-c91d-46aa-88f5-0d7cb8b03924-signing-key\") pod \"service-ca-7b994bc59-pllx6\" (UID: \"da8fbab5-c91d-46aa-88f5-0d7cb8b03924\") " pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.685315 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.685260 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da8fbab5-c91d-46aa-88f5-0d7cb8b03924-signing-cabundle\") pod \"service-ca-7b994bc59-pllx6\" (UID: \"da8fbab5-c91d-46aa-88f5-0d7cb8b03924\") " pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.786634 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.786555 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw9h7\" (UniqueName: \"kubernetes.io/projected/da8fbab5-c91d-46aa-88f5-0d7cb8b03924-kube-api-access-tw9h7\") pod \"service-ca-7b994bc59-pllx6\" (UID: \"da8fbab5-c91d-46aa-88f5-0d7cb8b03924\") " pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.786764 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.786710 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da8fbab5-c91d-46aa-88f5-0d7cb8b03924-signing-key\") pod \"service-ca-7b994bc59-pllx6\" (UID: \"da8fbab5-c91d-46aa-88f5-0d7cb8b03924\") " pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.786764 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.786744 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da8fbab5-c91d-46aa-88f5-0d7cb8b03924-signing-cabundle\") pod \"service-ca-7b994bc59-pllx6\" (UID: \"da8fbab5-c91d-46aa-88f5-0d7cb8b03924\") " pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.787432 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.787414 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da8fbab5-c91d-46aa-88f5-0d7cb8b03924-signing-cabundle\") pod \"service-ca-7b994bc59-pllx6\" (UID: \"da8fbab5-c91d-46aa-88f5-0d7cb8b03924\") " pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.789038 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.789022 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da8fbab5-c91d-46aa-88f5-0d7cb8b03924-signing-key\") pod \"service-ca-7b994bc59-pllx6\" (UID: \"da8fbab5-c91d-46aa-88f5-0d7cb8b03924\") " pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.797185 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.797164 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw9h7\" (UniqueName: \"kubernetes.io/projected/da8fbab5-c91d-46aa-88f5-0d7cb8b03924-kube-api-access-tw9h7\") pod \"service-ca-7b994bc59-pllx6\" (UID: \"da8fbab5-c91d-46aa-88f5-0d7cb8b03924\") " pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.844464 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.844434 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-pllx6" Jun 13 01:00:05.961169 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:05.961140 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-pllx6"] Jun 13 01:00:05.964654 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:05.964629 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda8fbab5_c91d_46aa_88f5_0d7cb8b03924.slice/crio-79f1154d6e71284c9fbb4014f06aa30aaef4340d40d62e4cacc222c3100c4578 WatchSource:0}: Error finding container 79f1154d6e71284c9fbb4014f06aa30aaef4340d40d62e4cacc222c3100c4578: Status 404 returned error can't find the container with id 79f1154d6e71284c9fbb4014f06aa30aaef4340d40d62e4cacc222c3100c4578 Jun 13 01:00:06.190168 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:06.190135 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 01:00:06.190597 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:06.190175 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 01:00:06.190597 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:06.190282 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 13 01:00:06.190597 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:06.190334 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls podName:210bbc79-fa75-47da-9a3e-69b702fa8cff nodeName:}" failed. No retries permitted until 2026-06-13 01:00:14.190320551 +0000 UTC m=+253.764249422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-2cjck" (UID: "210bbc79-fa75-47da-9a3e-69b702fa8cff") : secret "cluster-monitoring-operator-tls" not found Jun 13 01:00:06.190597 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:06.190369 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 13 01:00:06.190597 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:06.190438 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls podName:21318529-f4f2-46cf-808e-4a5786203157 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:14.190417911 +0000 UTC m=+253.764346786 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls") pod "cluster-samples-operator-9547488fd-gf8p5" (UID: "21318529-f4f2-46cf-808e-4a5786203157") : secret "samples-operator-tls" not found Jun 13 01:00:06.359625 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:06.359598 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dvssx_08c13986-e45f-4f66-a34e-0401741bdfcf/node-ca/0.log" Jun 13 01:00:06.392049 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:06.392020 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:06.392177 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:06.392064 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:06.392226 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:06.392175 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:14.392158172 +0000 UTC m=+253.966087039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : configmap references non-existent config key: service-ca.crt Jun 13 01:00:06.392226 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:06.392211 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 13 01:00:06.392313 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:06.392253 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs podName:c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:14.392239035 +0000 UTC m=+253.966167902 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs") pod "router-default-788bb8b696-bhgjk" (UID: "c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036") : secret "router-metrics-certs-default" not found Jun 13 01:00:06.539146 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:06.539066 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-pllx6" event={"ID":"da8fbab5-c91d-46aa-88f5-0d7cb8b03924","Type":"ContainerStarted","Data":"4b2ffed96df9d477386569a0425427859308081d56584c6dd76013059d77d17a"} Jun 13 01:00:06.539146 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:06.539120 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-pllx6" event={"ID":"da8fbab5-c91d-46aa-88f5-0d7cb8b03924","Type":"ContainerStarted","Data":"79f1154d6e71284c9fbb4014f06aa30aaef4340d40d62e4cacc222c3100c4578"} Jun 13 01:00:06.559726 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:06.559678 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7b994bc59-pllx6" podStartSLOduration=1.5596638779999998 podStartE2EDuration="1.559663878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:00:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:06.559001665 +0000 UTC m=+246.132930563" watchObservedRunningTime="2026-06-13 01:00:06.559663878 +0000 UTC m=+246.133592767" Jun 13 01:00:07.098303 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:07.098274 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:07.098547 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:07.098517 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 13 01:00:07.098660 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:07.098607 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 13 01:00:07.098660 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:07.098549 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:07.098660 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:07.098654 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:11.098619809 +0000 UTC m=+250.672548687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : secret "insights-runtime-extractor-tls" not found Jun 13 01:00:07.098857 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:07.098724 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:11.098709739 +0000 UTC m=+250.672638605 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : configmap "kube-rbac-proxy" not found Jun 13 01:00:07.361102 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:07.361016 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w4kkf_fa4f439a-83ef-4ba0-a9c6-ae5d1655d033/serve-healthcheck-canary/0.log" Jun 13 01:00:11.129655 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:11.129613 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:11.130119 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:11.129710 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:11.130119 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:11.129770 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 13 01:00:11.130119 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:11.129857 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:19.129841938 +0000 UTC m=+258.703770810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : secret "insights-runtime-extractor-tls" not found Jun 13 01:00:11.130119 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:11.129855 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 13 01:00:11.130119 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:11.129920 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm podName:fb25f6cb-295d-4c25-8614-f3e2a02d4538 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:19.129900369 +0000 UTC m=+258.703829237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kkm5c" (UID: "fb25f6cb-295d-4c25-8614-f3e2a02d4538") : configmap "kube-rbac-proxy" not found Jun 13 01:00:12.641682 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:12.641654 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 01:00:12.642022 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:12.641815 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 13 01:00:12.642022 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:12.641885 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs podName:7ae3796c-d916-4f7d-ac66-24899ca4e0f0 nodeName:}" failed. No retries permitted until 2026-06-13 01:02:14.641864969 +0000 UTC m=+374.215793839 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs") pod "network-metrics-daemon-ln4ft" (UID: "7ae3796c-d916-4f7d-ac66-24899ca4e0f0") : secret "metrics-daemon-secret" not found Jun 13 01:00:14.256447 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.256408 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 01:00:14.256447 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.256452 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 01:00:14.258753 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.258732 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21318529-f4f2-46cf-808e-4a5786203157-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-gf8p5\" (UID: \"21318529-f4f2-46cf-808e-4a5786203157\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 01:00:14.258845 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.258739 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/210bbc79-fa75-47da-9a3e-69b702fa8cff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-2cjck\" (UID: \"210bbc79-fa75-47da-9a3e-69b702fa8cff\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 01:00:14.370382 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.370357 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-29g65\"" Jun 13 01:00:14.375350 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.375332 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kg2ql\"" Jun 13 01:00:14.378186 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.378171 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" Jun 13 01:00:14.384098 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.384081 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" Jun 13 01:00:14.458213 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.458137 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:14.458213 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.458184 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:14.458935 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.458911 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-service-ca-bundle\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:14.460691 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.460671 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036-metrics-certs\") pod \"router-default-788bb8b696-bhgjk\" (UID: \"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036\") " pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:14.498532 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.498502 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-r6z2s\"" Jun 13 01:00:14.506422 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.506397 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:14.521294 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.521269 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck"] Jun 13 01:00:14.523230 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:14.523209 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210bbc79_fa75_47da_9a3e_69b702fa8cff.slice/crio-6cee8434f353b7ef3ad5fca9a7c1f9eb9f226b053f9d4f98eec42e089e42d146 WatchSource:0}: Error finding container 6cee8434f353b7ef3ad5fca9a7c1f9eb9f226b053f9d4f98eec42e089e42d146: Status 404 returned error can't find the container with id 6cee8434f353b7ef3ad5fca9a7c1f9eb9f226b053f9d4f98eec42e089e42d146 Jun 13 01:00:14.532931 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.532911 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5"] Jun 13 01:00:14.573323 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.573047 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" event={"ID":"210bbc79-fa75-47da-9a3e-69b702fa8cff","Type":"ContainerStarted","Data":"6cee8434f353b7ef3ad5fca9a7c1f9eb9f226b053f9d4f98eec42e089e42d146"} Jun 13 01:00:14.642130 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:14.642039 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-788bb8b696-bhgjk"] Jun 13 01:00:14.644481 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:14.644446 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9bcb0a1_3cc5_40e2_8d3c_30b54f1f3036.slice/crio-5ab92060cd4234c9cb050f55d2f052cbc3591ed026381a9c57698afe82e4b3e5 WatchSource:0}: Error finding container 5ab92060cd4234c9cb050f55d2f052cbc3591ed026381a9c57698afe82e4b3e5: Status 404 returned error can't find the container with id 5ab92060cd4234c9cb050f55d2f052cbc3591ed026381a9c57698afe82e4b3e5 Jun 13 01:00:15.580154 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:15.580041 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-788bb8b696-bhgjk" event={"ID":"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036","Type":"ContainerStarted","Data":"dc96977738f6d6198684a822db02a77d2188412f6d066b3dce9713c55d490aca"} Jun 13 01:00:15.580154 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:15.580112 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-788bb8b696-bhgjk" event={"ID":"c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036","Type":"ContainerStarted","Data":"5ab92060cd4234c9cb050f55d2f052cbc3591ed026381a9c57698afe82e4b3e5"} Jun 13 01:00:15.581760 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:15.581680 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" event={"ID":"21318529-f4f2-46cf-808e-4a5786203157","Type":"ContainerStarted","Data":"621fb6aff6fd55574e89975c545d50a397381da1565c658286e7407586fdb090"} Jun 13 01:00:15.606597 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:15.606540 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-788bb8b696-bhgjk" podStartSLOduration=17.606520397 podStartE2EDuration="17.606520397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:59:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:15.605434215 +0000 UTC m=+255.179363130" watchObservedRunningTime="2026-06-13 01:00:15.606520397 +0000 UTC m=+255.180449269" Jun 13 01:00:16.507582 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:16.507544 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:16.510680 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:16.510657 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:16.586974 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:16.586939 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:16.588540 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:16.588477 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-788bb8b696-bhgjk" Jun 13 01:00:17.591617 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:17.591581 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" event={"ID":"210bbc79-fa75-47da-9a3e-69b702fa8cff","Type":"ContainerStarted","Data":"ab0d36e13be2b42c5576804e6b661c83c257749a066e0a2e6d1226d70387281d"} Jun 13 01:00:17.593225 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:17.593199 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" event={"ID":"21318529-f4f2-46cf-808e-4a5786203157","Type":"ContainerStarted","Data":"d4453e5927924030cfae9c1cf0fa1e3a561a9c9e10e4551391798b296dad54ea"} Jun 13 01:00:17.593350 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:17.593247 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" event={"ID":"21318529-f4f2-46cf-808e-4a5786203157","Type":"ContainerStarted","Data":"40965f670aced920615a9eb6ec5dc1134014153e38eddb0aea5563247084156d"} Jun 13 01:00:17.610105 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:17.610058 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-2cjck" podStartSLOduration=17.547507988 podStartE2EDuration="19.6100421s" totalImagesPullingTime="2.062534112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:17.609291242 +0000 UTC m=+257.183220129" watchObservedRunningTime="2026-06-13 01:00:17.6100421 +0000 UTC m=+257.183970989" Jun 13 01:00:17.625741 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:17.625701 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-gf8p5" podStartSLOduration=17.61932046 podStartE2EDuration="19.62568797s" totalImagesPullingTime="2.00636751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:17.624692462 +0000 UTC m=+257.198621354" watchObservedRunningTime="2026-06-13 01:00:17.62568797 +0000 UTC m=+257.199616882" Jun 13 01:00:19.201362 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:19.201321 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:19.201761 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:19.201374 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:19.201935 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:19.201915 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb25f6cb-295d-4c25-8614-f3e2a02d4538-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:19.203553 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:19.203526 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb25f6cb-295d-4c25-8614-f3e2a02d4538-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kkm5c\" (UID: \"fb25f6cb-295d-4c25-8614-f3e2a02d4538\") " pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:19.282878 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:19.282843 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kkm5c" Jun 13 01:00:19.434546 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:19.434521 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kkm5c"] Jun 13 01:00:19.437003 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:19.436980 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb25f6cb_295d_4c25_8614_f3e2a02d4538.slice/crio-bd0dc55d5126af54dd3f327aa2194736ff6edb85aa04a5f7c594c255e981e509 WatchSource:0}: Error finding container bd0dc55d5126af54dd3f327aa2194736ff6edb85aa04a5f7c594c255e981e509: Status 404 returned error can't find the container with id bd0dc55d5126af54dd3f327aa2194736ff6edb85aa04a5f7c594c255e981e509 Jun 13 01:00:19.600919 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:19.600894 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kkm5c" event={"ID":"fb25f6cb-295d-4c25-8614-f3e2a02d4538","Type":"ContainerStarted","Data":"bd0dc55d5126af54dd3f327aa2194736ff6edb85aa04a5f7c594c255e981e509"} Jun 13 01:00:23.616051 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:23.616024 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kkm5c" event={"ID":"fb25f6cb-295d-4c25-8614-f3e2a02d4538","Type":"ContainerStarted","Data":"3fdc048cf9829aaf0f27ea8fb76ee0b26fcb6774a7563b085847de4aa1b82706"} Jun 13 01:00:23.616315 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:23.616079 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kkm5c" event={"ID":"fb25f6cb-295d-4c25-8614-f3e2a02d4538","Type":"ContainerStarted","Data":"c9a4f80c93627a5e79ba25e264a5b179b588d17354992d50ec55a76649dad9ac"} Jun 13 01:00:25.625414 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:25.625378 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kkm5c" event={"ID":"fb25f6cb-295d-4c25-8614-f3e2a02d4538","Type":"ContainerStarted","Data":"2a71a0a2827d011a78e997a994d3f65440842c096c697b51186edf186cf2b35f"} Jun 13 01:00:25.645453 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:25.645299 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kkm5c" podStartSLOduration=17.195604142 podStartE2EDuration="22.645280376s" totalImagesPullingTime="5.449676234s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 01:00:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:25.644805495 +0000 UTC m=+265.218734379" watchObservedRunningTime="2026-06-13 01:00:25.645280376 +0000 UTC m=+265.219209265" Jun 13 01:00:26.927297 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:26.927263 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-694c955b9-w2g79"] Jun 13 01:00:26.930561 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:26.930539 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-w2g79" Jun 13 01:00:26.933251 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:26.933229 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-b7smt\"" Jun 13 01:00:26.933564 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:26.933547 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 13 01:00:26.933649 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:26.933605 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 13 01:00:26.945645 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:26.945619 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-w2g79"] Jun 13 01:00:26.957933 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:26.957909 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chljb\" (UniqueName: \"kubernetes.io/projected/3e53de96-99fe-4974-bddf-4f033e866518-kube-api-access-chljb\") pod \"downloads-694c955b9-w2g79\" (UID: \"3e53de96-99fe-4974-bddf-4f033e866518\") " pod="openshift-console/downloads-694c955b9-w2g79" Jun 13 01:00:27.058372 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:27.058341 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chljb\" (UniqueName: \"kubernetes.io/projected/3e53de96-99fe-4974-bddf-4f033e866518-kube-api-access-chljb\") pod \"downloads-694c955b9-w2g79\" (UID: \"3e53de96-99fe-4974-bddf-4f033e866518\") " pod="openshift-console/downloads-694c955b9-w2g79" Jun 13 01:00:27.068458 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:27.068432 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chljb\" (UniqueName: \"kubernetes.io/projected/3e53de96-99fe-4974-bddf-4f033e866518-kube-api-access-chljb\") pod \"downloads-694c955b9-w2g79\" (UID: \"3e53de96-99fe-4974-bddf-4f033e866518\") " pod="openshift-console/downloads-694c955b9-w2g79" Jun 13 01:00:27.241947 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:27.241888 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-w2g79" Jun 13 01:00:27.368557 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:27.368530 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-w2g79"] Jun 13 01:00:27.371494 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:27.371463 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e53de96_99fe_4974_bddf_4f033e866518.slice/crio-119bbd66dbf48fe23f70d10621967c02ecf06378af5deb3601fca515f9ddb765 WatchSource:0}: Error finding container 119bbd66dbf48fe23f70d10621967c02ecf06378af5deb3601fca515f9ddb765: Status 404 returned error can't find the container with id 119bbd66dbf48fe23f70d10621967c02ecf06378af5deb3601fca515f9ddb765 Jun 13 01:00:27.632747 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:27.632713 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-w2g79" event={"ID":"3e53de96-99fe-4974-bddf-4f033e866518","Type":"ContainerStarted","Data":"119bbd66dbf48fe23f70d10621967c02ecf06378af5deb3601fca515f9ddb765"} Jun 13 01:00:29.170397 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.170367 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8"] Jun 13 01:00:29.173962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.173940 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.176862 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.176835 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 13 01:00:29.177023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.177000 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 13 01:00:29.178096 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.177946 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rqkmj\"" Jun 13 01:00:29.178096 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.177974 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 13 01:00:29.192381 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.192361 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8"] Jun 13 01:00:29.275537 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.275507 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7v49\" (UniqueName: \"kubernetes.io/projected/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-kube-api-access-b7v49\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.275655 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.275590 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.275655 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.275640 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-metrics-client-ca\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.275800 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.275696 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.376480 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.376445 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.376614 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.376512 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7v49\" (UniqueName: \"kubernetes.io/projected/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-kube-api-access-b7v49\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.376614 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.376579 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.376614 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:29.376604 2552 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 13 01:00:29.376813 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.376631 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-metrics-client-ca\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.376813 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:29.376678 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-prometheus-operator-tls podName:7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d nodeName:}" failed. No retries permitted until 2026-06-13 01:00:29.876658505 +0000 UTC m=+269.450587371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-prometheus-operator-tls") pod "prometheus-operator-864fcfc6f9-5d4z8" (UID: "7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d") : secret "prometheus-operator-tls" not found Jun 13 01:00:29.377418 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.377387 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-metrics-client-ca\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.379058 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.379038 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.385832 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.385768 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7v49\" (UniqueName: \"kubernetes.io/projected/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-kube-api-access-b7v49\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.880447 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.880416 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:29.883386 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:29.883358 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-5d4z8\" (UID: \"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:30.085728 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:30.085694 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" Jun 13 01:00:30.228229 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:30.228196 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8"] Jun 13 01:00:30.231639 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:30.231606 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a5a33f9_3cc9_4d5b_ab3f_aa100f256a4d.slice/crio-f29dc55fbc3c02730125b379d8e2b11038f1c4eb58ff35580dfbc5d7030238a6 WatchSource:0}: Error finding container f29dc55fbc3c02730125b379d8e2b11038f1c4eb58ff35580dfbc5d7030238a6: Status 404 returned error can't find the container with id f29dc55fbc3c02730125b379d8e2b11038f1c4eb58ff35580dfbc5d7030238a6 Jun 13 01:00:30.646010 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:30.645963 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" event={"ID":"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d","Type":"ContainerStarted","Data":"f29dc55fbc3c02730125b379d8e2b11038f1c4eb58ff35580dfbc5d7030238a6"} Jun 13 01:00:31.654143 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:31.654079 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" event={"ID":"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d","Type":"ContainerStarted","Data":"7d1cbd3c75565cc8ab7880827872036261c718c8ee52c8554e1b6dc112e664f0"} Jun 13 01:00:31.654143 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:31.654142 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" event={"ID":"7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d","Type":"ContainerStarted","Data":"813cf30ad416e4f1685042e957d0ecf572305ee73485fe6f1598999a67cfd903"} Jun 13 01:00:31.673948 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:31.673851 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-5d4z8" podStartSLOduration=1.483924602 podStartE2EDuration="2.673830867s" totalImagesPullingTime="1.189906265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:31.672370294 +0000 UTC m=+271.246299233" watchObservedRunningTime="2026-06-13 01:00:31.673830867 +0000 UTC m=+271.247759736" Jun 13 01:00:33.518275 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.518242 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-97l2g"] Jun 13 01:00:33.522047 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.522024 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.527519 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.524973 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 13 01:00:33.527519 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.525329 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 13 01:00:33.527519 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.525679 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5qpzh\"" Jun 13 01:00:33.527519 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.527002 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 13 01:00:33.614397 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.614216 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-tls\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.614397 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.614285 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.614397 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.614312 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-sys\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.614397 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.614353 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2p2m\" (UniqueName: \"kubernetes.io/projected/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-kube-api-access-x2p2m\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.614734 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.614438 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-textfile\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.614734 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.614476 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-root\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.614734 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.614530 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-wtmp\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.614734 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.614576 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-metrics-client-ca\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.614734 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.614600 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-accelerators-collector-config\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.716971 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-textfile\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717008 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-root\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717067 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-root\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717120 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-wtmp\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717164 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-metrics-client-ca\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717188 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-accelerators-collector-config\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717232 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-tls\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717282 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717304 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-sys\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717341 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2p2m\" (UniqueName: \"kubernetes.io/projected/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-kube-api-access-x2p2m\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717344 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-textfile\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:33.717465 2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:33.717516 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-tls podName:c91d0129-4b62-4ae5-a5d2-ba2d23846ec1 nodeName:}" failed. No retries permitted until 2026-06-13 01:00:34.217499405 +0000 UTC m=+273.791428277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-tls") pod "node-exporter-97l2g" (UID: "c91d0129-4b62-4ae5-a5d2-ba2d23846ec1") : secret "node-exporter-tls" not found Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717553 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-wtmp\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.717904 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-accelerators-collector-config\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.718040 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-metrics-client-ca\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.718941 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.718105 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-sys\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.724887 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.721080 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:33.732259 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:33.732236 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2p2m\" (UniqueName: \"kubernetes.io/projected/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-kube-api-access-x2p2m\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:34.221059 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.221019 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-tls\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:34.223959 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.223909 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c91d0129-4b62-4ae5-a5d2-ba2d23846ec1-node-exporter-tls\") pod \"node-exporter-97l2g\" (UID: \"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1\") " pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:34.436002 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.435975 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-97l2g" Jun 13 01:00:34.450077 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:34.450041 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91d0129_4b62_4ae5_a5d2_ba2d23846ec1.slice/crio-734dd7b62be251460c7e6bae8f7a9ccf194321ec31451bd23b89d4ec708e141c WatchSource:0}: Error finding container 734dd7b62be251460c7e6bae8f7a9ccf194321ec31451bd23b89d4ec708e141c: Status 404 returned error can't find the container with id 734dd7b62be251460c7e6bae8f7a9ccf194321ec31451bd23b89d4ec708e141c Jun 13 01:00:34.587561 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.587521 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 13 01:00:34.594485 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.594087 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.596652 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.596628 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 13 01:00:34.596759 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.596718 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 13 01:00:34.597139 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.596889 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 13 01:00:34.597263 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.596973 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 13 01:00:34.597263 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.597025 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 13 01:00:34.597370 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.597044 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 13 01:00:34.597370 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.597082 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 13 01:00:34.597474 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.597102 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 13 01:00:34.597474 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.597117 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kbjkr\"" Jun 13 01:00:34.597678 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.597661 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 13 01:00:34.606683 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.606662 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 13 01:00:34.624850 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.624823 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.624938 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.624889 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.624938 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.624913 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625026 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.624947 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625026 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.624970 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-web-config\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625026 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.624995 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-config-out\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625141 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.625051 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625182 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.625123 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625227 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.625168 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625227 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.625206 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625312 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.625245 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625312 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.625281 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwk26\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-kube-api-access-bwk26\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.625372 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.625335 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-config-volume\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.669664 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.669631 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-97l2g" event={"ID":"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1","Type":"ContainerStarted","Data":"734dd7b62be251460c7e6bae8f7a9ccf194321ec31451bd23b89d4ec708e141c"} Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726437 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726501 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726527 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726559 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:34.726654 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-trusted-ca-bundle podName:a50a0364-2341-441e-b274-ad9db6ea659a nodeName:}" failed. No retries permitted until 2026-06-13 01:00:35.226636579 +0000 UTC m=+274.800565465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a") : configmap references non-existent config key: ca-bundle.crt Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726674 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-web-config\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726704 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-config-out\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726749 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726817 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726848 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726874 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726912 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.726946 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwk26\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-kube-api-access-bwk26\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727754 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.727342 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.727977 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.727956 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.729827 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.729804 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.729909 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.729893 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-config-volume\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.730551 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.730465 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-web-config\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.731356 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.731301 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.732638 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.731904 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.732638 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.732202 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.732638 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.732581 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.733572 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.733197 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.733572 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.733539 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-config-volume\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.734614 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.734577 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-config-out\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:34.742386 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:34.742339 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwk26\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-kube-api-access-bwk26\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:35.235403 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:35.235365 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:35.236386 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:35.236336 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:35.511904 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:35.511824 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:00:35.761214 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:35.761185 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 13 01:00:35.863773 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:35.863742 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50a0364_2341_441e_b274_ad9db6ea659a.slice/crio-dd34bc86c51d093f86568864a879a8e81e1fbe5294026f25c75bdb6e451fffdf WatchSource:0}: Error finding container dd34bc86c51d093f86568864a879a8e81e1fbe5294026f25c75bdb6e451fffdf: Status 404 returned error can't find the container with id dd34bc86c51d093f86568864a879a8e81e1fbe5294026f25c75bdb6e451fffdf Jun 13 01:00:36.681965 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:36.681931 2552 generic.go:356] "Generic (PLEG): container finished" podID="c91d0129-4b62-4ae5-a5d2-ba2d23846ec1" containerID="02cb531a967cf92e7e83d6e9791cff2f13730873c84c0d6de9f16baf23f3719a" exitCode=0 Jun 13 01:00:36.682252 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:36.682052 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-97l2g" event={"ID":"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1","Type":"ContainerDied","Data":"02cb531a967cf92e7e83d6e9791cff2f13730873c84c0d6de9f16baf23f3719a"} Jun 13 01:00:36.684103 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:36.684077 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerStarted","Data":"dd34bc86c51d093f86568864a879a8e81e1fbe5294026f25c75bdb6e451fffdf"} Jun 13 01:00:37.690079 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:37.690043 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-97l2g" event={"ID":"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1","Type":"ContainerStarted","Data":"37faf7e3d155e50cb577fb1560bd8af21e9f84db626cab65795c7a27c73c7d84"} Jun 13 01:00:39.784799 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.784748 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 13 01:00:39.787901 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.787882 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.793703 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.793678 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 13 01:00:39.793839 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.793805 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4l7xx\"" Jun 13 01:00:39.795214 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.795164 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 13 01:00:39.795333 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.795234 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 13 01:00:39.795333 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.795169 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-agog0fucjmlfu\"" Jun 13 01:00:39.796248 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.796229 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 13 01:00:39.796248 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.796241 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 13 01:00:39.796446 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.796431 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 13 01:00:39.797146 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.797126 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 13 01:00:39.800405 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.799251 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 13 01:00:39.800405 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.799442 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 13 01:00:39.800405 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.799869 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 13 01:00:39.800405 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.800111 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 13 01:00:39.800732 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.800578 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 13 01:00:39.800732 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.800635 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 13 01:00:39.821422 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.821402 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 13 01:00:39.827733 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.827651 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 13 01:00:39.845391 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.845358 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 13 01:00:39.879539 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.879452 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.879665 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.879614 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.879665 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.879646 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.879772 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.879670 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.879772 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.879736 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.879907 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.879844 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8gc9\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-kube-api-access-s8gc9\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.879965 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.879910 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.879965 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.879940 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880066 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.879976 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880066 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880005 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880066 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880033 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880190 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880123 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880190 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880168 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880190 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880189 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880326 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880204 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880326 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880230 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880326 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880268 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880326 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880301 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880475 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880326 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.880475 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.880352 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.981865 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.981707 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.981865 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.981760 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.981865 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.981809 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982218 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982189 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982379 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982358 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982465 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982398 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982465 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982421 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982465 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982448 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982471 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982494 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982522 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982553 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982561 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982580 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982603 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.982626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.982626 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.984292 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.983589 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.984609 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.984496 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.986608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.984792 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.986608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.984836 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.986608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.984864 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.986608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.984904 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.986608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.984941 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8gc9\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-kube-api-access-s8gc9\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.986608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.985154 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.986608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.985819 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.986608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.986483 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.987721 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.987412 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.987721 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.987414 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.988321 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.987986 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.988321 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.988215 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.988321 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.988234 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.988321 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.988282 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.988553 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.988330 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.988553 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.988524 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.989481 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.989074 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.989481 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.989441 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.990655 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.990615 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.990964 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.990928 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.992757 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.992724 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:39.993699 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:39.993678 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8gc9\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-kube-api-access-s8gc9\") pod \"prometheus-k8s-0\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:40.101330 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:40.101274 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:00:41.300615 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:41.300562 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" podUID="7140ef67-07de-4771-b594-0a3c63f7254c" Jun 13 01:00:41.301075 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:41.300894 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" podUID="dfa7355d-fa80-4074-8574-844ade0436f7" Jun 13 01:00:41.301075 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:41.300906 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-f5lsk" podUID="d35bfb84-8978-47a6-8c05-034fb69773dd" Jun 13 01:00:41.704816 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:41.704733 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 01:00:41.704956 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:41.704735 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f5lsk" Jun 13 01:00:41.704956 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:41.704735 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 01:00:44.225154 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.225131 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 01:00:44.225478 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.225242 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 01:00:44.228381 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.228021 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7140ef67-07de-4771-b594-0a3c63f7254c-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-tq4xr\" (UID: \"7140ef67-07de-4771-b594-0a3c63f7254c\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 01:00:44.229663 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.229643 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"image-registry-869f996f96-dnd7g\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 01:00:44.313533 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.313505 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 13 01:00:44.325758 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.325728 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 01:00:44.327795 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.327761 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d35bfb84-8978-47a6-8c05-034fb69773dd-metrics-tls\") pod \"dns-default-f5lsk\" (UID: \"d35bfb84-8978-47a6-8c05-034fb69773dd\") " pod="openshift-dns/dns-default-f5lsk" Jun 13 01:00:44.338639 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:44.338611 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6129c3f_a643_4ded_95f3_3e91e4fa79f9.slice/crio-3d3f41f2a2501dac3e0b3a29271f3d883c4c65d64e0729a62778693a11ee8c66 WatchSource:0}: Error finding container 3d3f41f2a2501dac3e0b3a29271f3d883c4c65d64e0729a62778693a11ee8c66: Status 404 returned error can't find the container with id 3d3f41f2a2501dac3e0b3a29271f3d883c4c65d64e0729a62778693a11ee8c66 Jun 13 01:00:44.408800 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.408451 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wxfsc\"" Jun 13 01:00:44.408800 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.408724 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7d8x8\"" Jun 13 01:00:44.408970 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.408882 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8gq9q\"" Jun 13 01:00:44.417008 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.416991 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 01:00:44.417098 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.417082 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" Jun 13 01:00:44.417142 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.417081 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f5lsk" Jun 13 01:00:44.575421 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.574990 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-tq4xr"] Jun 13 01:00:44.579468 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:00:44.579317 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7140ef67_07de_4771_b594_0a3c63f7254c.slice/crio-759cf915c4bd850e6f73626356508df71f7448ca4b6e8488c1c313a070a0ccb6 WatchSource:0}: Error finding container 759cf915c4bd850e6f73626356508df71f7448ca4b6e8488c1c313a070a0ccb6: Status 404 returned error can't find the container with id 759cf915c4bd850e6f73626356508df71f7448ca4b6e8488c1c313a070a0ccb6 Jun 13 01:00:44.719451 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.719109 2552 generic.go:356] "Generic (PLEG): container finished" podID="a50a0364-2341-441e-b274-ad9db6ea659a" containerID="a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85" exitCode=0 Jun 13 01:00:44.719451 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.719311 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerDied","Data":"a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85"} Jun 13 01:00:44.723328 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.723293 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-97l2g" event={"ID":"c91d0129-4b62-4ae5-a5d2-ba2d23846ec1","Type":"ContainerStarted","Data":"6d16bb839f934a4c3d7411ec9de5edde6671a1df1fbd2e0d455577ef29b5aeb8"} Jun 13 01:00:44.725800 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.725741 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" event={"ID":"7140ef67-07de-4771-b594-0a3c63f7254c","Type":"ContainerStarted","Data":"759cf915c4bd850e6f73626356508df71f7448ca4b6e8488c1c313a070a0ccb6"} Jun 13 01:00:44.728247 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.728222 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-w2g79" event={"ID":"3e53de96-99fe-4974-bddf-4f033e866518","Type":"ContainerStarted","Data":"47d3587dafbf02abd2faf9c592ddcdbebe650ff2ad27faa3c1974708e89aa994"} Jun 13 01:00:44.728363 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.728268 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-694c955b9-w2g79" Jun 13 01:00:44.730148 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.730127 2552 generic.go:356] "Generic (PLEG): container finished" podID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerID="f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41" exitCode=0 Jun 13 01:00:44.730741 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.730304 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerDied","Data":"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41"} Jun 13 01:00:44.730870 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.730801 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerStarted","Data":"3d3f41f2a2501dac3e0b3a29271f3d883c4c65d64e0729a62778693a11ee8c66"} Jun 13 01:00:44.742902 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.742881 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-694c955b9-w2g79" Jun 13 01:00:44.769102 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.769051 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-694c955b9-w2g79" podStartSLOduration=1.928426289 podStartE2EDuration="18.769037356s" totalImagesPullingTime="16.840611067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:44.767071372 +0000 UTC m=+284.341000260" watchObservedRunningTime="2026-06-13 01:00:44.769037356 +0000 UTC m=+284.342966244" Jun 13 01:00:44.795992 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.794743 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-97l2g" podStartSLOduration=10.258278992 podStartE2EDuration="11.794722021s" totalImagesPullingTime="1.428869834s" totalInitContainerRuntime="107.573195ms" isStatefulPod=true podCreationTimestamp="2026-06-13 01:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:44.792942807 +0000 UTC m=+284.366871696" watchObservedRunningTime="2026-06-13 01:00:44.794722021 +0000 UTC m=+284.368650910" Jun 13 01:00:44.820413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.819851 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f5lsk"] Jun 13 01:00:44.825487 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:44.824988 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-869f996f96-dnd7g"] Jun 13 01:00:45.754811 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:45.753816 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" event={"ID":"dfa7355d-fa80-4074-8574-844ade0436f7","Type":"ContainerStarted","Data":"cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec"} Jun 13 01:00:45.754811 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:45.753886 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" event={"ID":"dfa7355d-fa80-4074-8574-844ade0436f7","Type":"ContainerStarted","Data":"f14c5ff49d9644a5ff0de718c1bebb93c0553cd1f32ad5599e3d92f126ae7095"} Jun 13 01:00:45.754811 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:45.754741 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 01:00:45.758860 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:45.758827 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f5lsk" event={"ID":"d35bfb84-8978-47a6-8c05-034fb69773dd","Type":"ContainerStarted","Data":"6274c50358b123eaf8d54077235efb942598fe99c11c9f1ce46f90a80c51dfa5"} Jun 13 01:00:45.781344 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:45.781294 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" podStartSLOduration=284.781274526 podStartE2EDuration="4m44.781274526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:45.779583716 +0000 UTC m=+285.353512605" watchObservedRunningTime="2026-06-13 01:00:45.781274526 +0000 UTC m=+285.355203397" Jun 13 01:00:50.789467 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.789418 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f5lsk" event={"ID":"d35bfb84-8978-47a6-8c05-034fb69773dd","Type":"ContainerStarted","Data":"1bc950bc1a3b41595283bf398493c0db59fefa9a8d90680d22b749b84066b5f4"} Jun 13 01:00:50.790390 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.789521 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f5lsk" event={"ID":"d35bfb84-8978-47a6-8c05-034fb69773dd","Type":"ContainerStarted","Data":"fce1b83af890fb15a2ce4dcaa05729d219a87ad72a6d68106aea572a2ca62696"} Jun 13 01:00:50.790390 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.789558 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f5lsk" Jun 13 01:00:50.792044 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.792014 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerStarted","Data":"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b"} Jun 13 01:00:50.792159 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.792087 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerStarted","Data":"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718"} Jun 13 01:00:50.795815 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.795773 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerStarted","Data":"a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8"} Jun 13 01:00:50.795928 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.795862 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerStarted","Data":"5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5"} Jun 13 01:00:50.795928 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.795907 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerStarted","Data":"e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5"} Jun 13 01:00:50.796040 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.795949 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerStarted","Data":"f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284"} Jun 13 01:00:50.796040 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.795986 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerStarted","Data":"1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03"} Jun 13 01:00:50.797408 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.797359 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" event={"ID":"7140ef67-07de-4771-b594-0a3c63f7254c","Type":"ContainerStarted","Data":"f110a867547d82de49ded1b1705c23dd5c3e8369b743c58a6628dcc3b30f6521"} Jun 13 01:00:50.808320 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:50.808275 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f5lsk" podStartSLOduration=251.718066927 podStartE2EDuration="4m16.808261796s" totalImagesPullingTime="5.090194869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:50.806052808 +0000 UTC m=+290.379981725" watchObservedRunningTime="2026-06-13 01:00:50.808261796 +0000 UTC m=+290.382190696" Jun 13 01:00:53.815051 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:53.815019 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerStarted","Data":"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a"} Jun 13 01:00:53.815511 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:53.815085 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerStarted","Data":"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea"} Jun 13 01:00:53.815511 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:53.815129 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerStarted","Data":"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22"} Jun 13 01:00:53.815511 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:53.815165 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerStarted","Data":"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea"} Jun 13 01:00:53.818189 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:53.818167 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerStarted","Data":"34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d"} Jun 13 01:00:53.850562 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:53.848765 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-867b76cb74-tq4xr" podStartSLOduration=287.521749949 podStartE2EDuration="4m52.84874719s" totalImagesPullingTime="5.326997241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:50.822860358 +0000 UTC m=+290.396789247" watchObservedRunningTime="2026-06-13 01:00:53.84874719 +0000 UTC m=+293.422676078" Jun 13 01:00:53.850562 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:53.850262 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.582636797 podStartE2EDuration="14.850246025s" totalImagesPullingTime="8.249257255s" totalInitContainerRuntime="18.351973ms" isStatefulPod=false podCreationTimestamp="2026-06-13 01:00:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:53.846844621 +0000 UTC m=+293.420773509" watchObservedRunningTime="2026-06-13 01:00:53.850246025 +0000 UTC m=+293.424174918" Jun 13 01:00:53.875696 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:53.875647 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.219326801 podStartE2EDuration="19.875634958s" totalImagesPullingTime="15.636515599s" totalInitContainerRuntime="19.792558ms" isStatefulPod=false podCreationTimestamp="2026-06-13 01:00:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:00:53.875195779 +0000 UTC m=+293.449124699" watchObservedRunningTime="2026-06-13 01:00:53.875634958 +0000 UTC m=+293.449563845" Jun 13 01:00:54.850732 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:00:54.850696 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ln4ft" podUID="7ae3796c-d916-4f7d-ac66-24899ca4e0f0" Jun 13 01:00:55.102383 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:00:55.102304 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:00.807886 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:00.807855 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f5lsk" Jun 13 01:01:00.813448 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:00.813426 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:01:00.813881 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:00.813858 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:01:00.820214 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:00.820199 2552 kubelet.go:1648] "Image garbage collection succeeded" Jun 13 01:01:04.421608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:04.421565 2552 patch_prober.go:28] interesting pod/image-registry-869f996f96-dnd7g container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 13 01:01:04.461093 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:04.421636 2552 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" podUID="dfa7355d-fa80-4074-8574-844ade0436f7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 13 01:01:07.776747 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:07.776719 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 01:01:09.849671 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:09.849635 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 01:01:12.896319 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:12.896288 2552 generic.go:356] "Generic (PLEG): container finished" podID="f7edddc5-6c88-4ba0-9165-9bb0e62220a1" containerID="62b91e8426c0f0a11c09b447f85b8be4f1045a0e3d12cc7d1c256a4b2c0fabbe" exitCode=0 Jun 13 01:01:12.896775 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:12.896356 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" event={"ID":"f7edddc5-6c88-4ba0-9165-9bb0e62220a1","Type":"ContainerDied","Data":"62b91e8426c0f0a11c09b447f85b8be4f1045a0e3d12cc7d1c256a4b2c0fabbe"} Jun 13 01:01:12.896775 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:12.896677 2552 scope.go:117] "RemoveContainer" containerID="62b91e8426c0f0a11c09b447f85b8be4f1045a0e3d12cc7d1c256a4b2c0fabbe" Jun 13 01:01:13.904282 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:13.904247 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-q9jvp" event={"ID":"f7edddc5-6c88-4ba0-9165-9bb0e62220a1","Type":"ContainerStarted","Data":"de22f6ad109536ea356564318fde3536d81a317231cf62aa4f7c7e6cdb9f4c5e"} Jun 13 01:01:13.905573 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:13.905547 2552 generic.go:356] "Generic (PLEG): container finished" podID="3c53466a-47d5-4367-ba1f-de324cb84360" containerID="626b835b575a00fae457595fb3b2498e080cccfea52e2dc0a774cfadbffaff8c" exitCode=0 Jun 13 01:01:13.905695 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:13.905617 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" event={"ID":"3c53466a-47d5-4367-ba1f-de324cb84360","Type":"ContainerDied","Data":"626b835b575a00fae457595fb3b2498e080cccfea52e2dc0a774cfadbffaff8c"} Jun 13 01:01:13.905927 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:13.905913 2552 scope.go:117] "RemoveContainer" containerID="626b835b575a00fae457595fb3b2498e080cccfea52e2dc0a774cfadbffaff8c" Jun 13 01:01:14.588872 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:14.588848 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f5lsk_d35bfb84-8978-47a6-8c05-034fb69773dd/dns/0.log" Jun 13 01:01:14.595085 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:14.595066 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f5lsk_d35bfb84-8978-47a6-8c05-034fb69773dd/kube-rbac-proxy/0.log" Jun 13 01:01:14.910597 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:14.910516 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-7f6mh" event={"ID":"3c53466a-47d5-4367-ba1f-de324cb84360","Type":"ContainerStarted","Data":"6cee83f3713af269ab49985f2b079719a287122ac6dc3037d235e21a02b2f6c4"} Jun 13 01:01:15.093433 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:15.093405 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xf2jd_87376b5e-f663-4b36-affd-55da108f6af7/dns-node-resolver/0.log" Jun 13 01:01:17.296216 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:17.296186 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-869f996f96-dnd7g"] Jun 13 01:01:17.923102 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:17.923019 2552 generic.go:356] "Generic (PLEG): container finished" podID="e85773c6-ab14-45ea-8df1-20975bf51131" containerID="2ce9805cc3bf3feae5916735ab977d6f197be74008cb5626952ce6fad13a11d1" exitCode=0 Jun 13 01:01:17.923271 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:17.923096 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" event={"ID":"e85773c6-ab14-45ea-8df1-20975bf51131","Type":"ContainerDied","Data":"2ce9805cc3bf3feae5916735ab977d6f197be74008cb5626952ce6fad13a11d1"} Jun 13 01:01:17.923483 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:17.923466 2552 scope.go:117] "RemoveContainer" containerID="2ce9805cc3bf3feae5916735ab977d6f197be74008cb5626952ce6fad13a11d1" Jun 13 01:01:18.929439 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:18.929406 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-lchjn" event={"ID":"e85773c6-ab14-45ea-8df1-20975bf51131","Type":"ContainerStarted","Data":"72dbfa60eca94453623770c74edbbc6b793029a9406bf510492269d57ac0ac1a"} Jun 13 01:01:40.101909 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:40.101879 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:40.117594 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:40.117567 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:41.028551 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:41.028525 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:42.316045 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:42.315931 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" podUID="dfa7355d-fa80-4074-8574-844ade0436f7" containerName="registry" containerID="cri-o://cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec" gracePeriod=30 Jun 13 01:01:43.564147 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.564121 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 01:01:43.650891 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.650828 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-registry-certificates\") pod \"dfa7355d-fa80-4074-8574-844ade0436f7\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " Jun 13 01:01:43.650891 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.650864 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-image-registry-private-configuration\") pod \"dfa7355d-fa80-4074-8574-844ade0436f7\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " Jun 13 01:01:43.651074 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.650938 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfa7355d-fa80-4074-8574-844ade0436f7-ca-trust-extracted\") pod \"dfa7355d-fa80-4074-8574-844ade0436f7\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " Jun 13 01:01:43.651074 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.650970 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-bound-sa-token\") pod \"dfa7355d-fa80-4074-8574-844ade0436f7\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " Jun 13 01:01:43.651074 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.651003 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-trusted-ca\") pod \"dfa7355d-fa80-4074-8574-844ade0436f7\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " Jun 13 01:01:43.651074 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.651043 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2rh5\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-kube-api-access-d2rh5\") pod \"dfa7355d-fa80-4074-8574-844ade0436f7\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " Jun 13 01:01:43.651286 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.651078 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-installation-pull-secrets\") pod \"dfa7355d-fa80-4074-8574-844ade0436f7\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " Jun 13 01:01:43.651286 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.651112 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") pod \"dfa7355d-fa80-4074-8574-844ade0436f7\" (UID: \"dfa7355d-fa80-4074-8574-844ade0436f7\") " Jun 13 01:01:43.651286 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.651230 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "dfa7355d-fa80-4074-8574-844ade0436f7" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:43.651443 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.651397 2552 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-registry-certificates\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:43.651526 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.651499 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "dfa7355d-fa80-4074-8574-844ade0436f7" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:43.653420 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.653395 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "dfa7355d-fa80-4074-8574-844ade0436f7" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:43.653641 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.653424 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "dfa7355d-fa80-4074-8574-844ade0436f7" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:01:43.653756 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.653703 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-kube-api-access-d2rh5" (OuterVolumeSpecName: "kube-api-access-d2rh5") pod "dfa7355d-fa80-4074-8574-844ade0436f7" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7"). InnerVolumeSpecName "kube-api-access-d2rh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:01:43.653756 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.653718 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "dfa7355d-fa80-4074-8574-844ade0436f7" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:43.653756 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.653704 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "dfa7355d-fa80-4074-8574-844ade0436f7" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:01:43.659259 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.659235 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa7355d-fa80-4074-8574-844ade0436f7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "dfa7355d-fa80-4074-8574-844ade0436f7" (UID: "dfa7355d-fa80-4074-8574-844ade0436f7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:01:43.752657 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.752632 2552 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfa7355d-fa80-4074-8574-844ade0436f7-ca-trust-extracted\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:43.752657 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.752654 2552 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-bound-sa-token\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:43.752816 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.752669 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfa7355d-fa80-4074-8574-844ade0436f7-trusted-ca\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:43.752816 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.752681 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2rh5\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-kube-api-access-d2rh5\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:43.752816 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.752693 2552 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-installation-pull-secrets\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:43.752816 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.752705 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfa7355d-fa80-4074-8574-844ade0436f7-registry-tls\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:43.752816 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:43.752719 2552 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfa7355d-fa80-4074-8574-844ade0436f7-image-registry-private-configuration\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:44.027038 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.026969 2552 generic.go:356] "Generic (PLEG): container finished" podID="dfa7355d-fa80-4074-8574-844ade0436f7" containerID="cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec" exitCode=0 Jun 13 01:01:44.027038 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.027014 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" event={"ID":"dfa7355d-fa80-4074-8574-844ade0436f7","Type":"ContainerDied","Data":"cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec"} Jun 13 01:01:44.027178 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.027046 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" event={"ID":"dfa7355d-fa80-4074-8574-844ade0436f7","Type":"ContainerDied","Data":"f14c5ff49d9644a5ff0de718c1bebb93c0553cd1f32ad5599e3d92f126ae7095"} Jun 13 01:01:44.027178 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.027015 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-869f996f96-dnd7g" Jun 13 01:01:44.028007 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.027992 2552 scope.go:117] "RemoveContainer" containerID="cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec" Jun 13 01:01:44.037294 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.037276 2552 scope.go:117] "RemoveContainer" containerID="cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec" Jun 13 01:01:44.037542 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:44.037524 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec\": container with ID starting with cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec not found: ID does not exist" containerID="cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec" Jun 13 01:01:44.037588 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.037549 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec"} err="failed to get container status \"cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec\": rpc error: code = NotFound desc = could not find container \"cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec\": container with ID starting with cb50d86329a1fd8163a9294f5d30ef8af481c0900ea80056562b4bc1d6ec59ec not found: ID does not exist" Jun 13 01:01:44.049747 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.049725 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-869f996f96-dnd7g"] Jun 13 01:01:44.054053 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.054032 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-869f996f96-dnd7g"] Jun 13 01:01:44.854336 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:44.854303 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfa7355d-fa80-4074-8574-844ade0436f7" path="/var/lib/kubelet/pods/dfa7355d-fa80-4074-8574-844ade0436f7/volumes" Jun 13 01:01:53.825066 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:53.825035 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 13 01:01:53.825418 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:53.825352 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="alertmanager" containerID="cri-o://1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03" gracePeriod=120 Jun 13 01:01:53.825480 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:53.825421 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy-metric" containerID="cri-o://a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8" gracePeriod=120 Jun 13 01:01:53.825534 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:53.825452 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="prom-label-proxy" containerID="cri-o://34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d" gracePeriod=120 Jun 13 01:01:53.825534 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:53.825481 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="config-reloader" containerID="cri-o://f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284" gracePeriod=120 Jun 13 01:01:53.825534 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:53.825480 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy" containerID="cri-o://5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5" gracePeriod=120 Jun 13 01:01:53.825672 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:53.825452 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy-web" containerID="cri-o://e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5" gracePeriod=120 Jun 13 01:01:54.075768 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075695 2552 generic.go:356] "Generic (PLEG): container finished" podID="a50a0364-2341-441e-b274-ad9db6ea659a" containerID="34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d" exitCode=0 Jun 13 01:01:54.075768 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075717 2552 generic.go:356] "Generic (PLEG): container finished" podID="a50a0364-2341-441e-b274-ad9db6ea659a" containerID="a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8" exitCode=0 Jun 13 01:01:54.075768 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075723 2552 generic.go:356] "Generic (PLEG): container finished" podID="a50a0364-2341-441e-b274-ad9db6ea659a" containerID="5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5" exitCode=0 Jun 13 01:01:54.075768 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075729 2552 generic.go:356] "Generic (PLEG): container finished" podID="a50a0364-2341-441e-b274-ad9db6ea659a" containerID="f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284" exitCode=0 Jun 13 01:01:54.075768 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075735 2552 generic.go:356] "Generic (PLEG): container finished" podID="a50a0364-2341-441e-b274-ad9db6ea659a" containerID="1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03" exitCode=0 Jun 13 01:01:54.076010 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075768 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerDied","Data":"34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d"} Jun 13 01:01:54.076010 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075853 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerDied","Data":"a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8"} Jun 13 01:01:54.076010 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075885 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerDied","Data":"5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5"} Jun 13 01:01:54.076010 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075910 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerDied","Data":"f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284"} Jun 13 01:01:54.076010 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:54.075934 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerDied","Data":"1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03"} Jun 13 01:01:55.072522 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.072499 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:01:55.083374 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.083351 2552 generic.go:356] "Generic (PLEG): container finished" podID="a50a0364-2341-441e-b274-ad9db6ea659a" containerID="e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5" exitCode=0 Jun 13 01:01:55.083467 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.083400 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerDied","Data":"e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5"} Jun 13 01:01:55.083467 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.083426 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 13 01:01:55.083533 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.083468 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a50a0364-2341-441e-b274-ad9db6ea659a","Type":"ContainerDied","Data":"dd34bc86c51d093f86568864a879a8e81e1fbe5294026f25c75bdb6e451fffdf"} Jun 13 01:01:55.083533 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.083518 2552 scope.go:117] "RemoveContainer" containerID="34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d" Jun 13 01:01:55.090790 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.090745 2552 scope.go:117] "RemoveContainer" containerID="a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8" Jun 13 01:01:55.097575 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.097556 2552 scope.go:117] "RemoveContainer" containerID="5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5" Jun 13 01:01:55.105945 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.105925 2552 scope.go:117] "RemoveContainer" containerID="e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5" Jun 13 01:01:55.115432 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.115413 2552 scope.go:117] "RemoveContainer" containerID="f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284" Jun 13 01:01:55.123675 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.123659 2552 scope.go:117] "RemoveContainer" containerID="1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03" Jun 13 01:01:55.132679 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.132657 2552 scope.go:117] "RemoveContainer" containerID="a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85" Jun 13 01:01:55.140437 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.140423 2552 scope.go:117] "RemoveContainer" containerID="34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d" Jun 13 01:01:55.140672 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:55.140651 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d\": container with ID starting with 34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d not found: ID does not exist" containerID="34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d" Jun 13 01:01:55.140744 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.140684 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d"} err="failed to get container status \"34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d\": rpc error: code = NotFound desc = could not find container \"34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d\": container with ID starting with 34116db04891782f17e3fb6a953933cb139b6f012cb911c57e6306ace887814d not found: ID does not exist" Jun 13 01:01:55.140744 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.140710 2552 scope.go:117] "RemoveContainer" containerID="a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8" Jun 13 01:01:55.140986 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:55.140969 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8\": container with ID starting with a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8 not found: ID does not exist" containerID="a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8" Jun 13 01:01:55.141055 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.140992 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8"} err="failed to get container status \"a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8\": rpc error: code = NotFound desc = could not find container \"a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8\": container with ID starting with a74025fd46f792f8ed4f48d68570966bb564c3b6086b0dca885003d7ac1291e8 not found: ID does not exist" Jun 13 01:01:55.141055 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.141009 2552 scope.go:117] "RemoveContainer" containerID="5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5" Jun 13 01:01:55.141281 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:55.141252 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5\": container with ID starting with 5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5 not found: ID does not exist" containerID="5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5" Jun 13 01:01:55.141328 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.141288 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5"} err="failed to get container status \"5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5\": rpc error: code = NotFound desc = could not find container \"5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5\": container with ID starting with 5684fd472d91449ed76ebef34207aaa6261b5e2165287e26dec4132cf9f365a5 not found: ID does not exist" Jun 13 01:01:55.141328 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.141303 2552 scope.go:117] "RemoveContainer" containerID="e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5" Jun 13 01:01:55.141530 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:55.141516 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5\": container with ID starting with e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5 not found: ID does not exist" containerID="e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5" Jun 13 01:01:55.141567 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.141533 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5"} err="failed to get container status \"e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5\": rpc error: code = NotFound desc = could not find container \"e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5\": container with ID starting with e3b2ba9a10a9eb489f662019853ef140723c79695407dc40392d257758820ee5 not found: ID does not exist" Jun 13 01:01:55.141567 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.141544 2552 scope.go:117] "RemoveContainer" containerID="f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284" Jun 13 01:01:55.141751 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:55.141737 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284\": container with ID starting with f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284 not found: ID does not exist" containerID="f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284" Jun 13 01:01:55.141874 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.141753 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284"} err="failed to get container status \"f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284\": rpc error: code = NotFound desc = could not find container \"f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284\": container with ID starting with f5651374d9c8217924dfe1f6c180ceea04b7f3c5feb56d2bda2319c9aa3af284 not found: ID does not exist" Jun 13 01:01:55.141874 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.141764 2552 scope.go:117] "RemoveContainer" containerID="1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03" Jun 13 01:01:55.142122 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:55.142107 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03\": container with ID starting with 1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03 not found: ID does not exist" containerID="1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03" Jun 13 01:01:55.142167 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.142125 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03"} err="failed to get container status \"1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03\": rpc error: code = NotFound desc = could not find container \"1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03\": container with ID starting with 1a11310c604a147a0293509f730e117edb583a37f6a23f47c6e3658c5ed54e03 not found: ID does not exist" Jun 13 01:01:55.142167 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.142139 2552 scope.go:117] "RemoveContainer" containerID="a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85" Jun 13 01:01:55.142362 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:55.142343 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85\": container with ID starting with a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85 not found: ID does not exist" containerID="a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85" Jun 13 01:01:55.142447 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.142365 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85"} err="failed to get container status \"a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85\": rpc error: code = NotFound desc = could not find container \"a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85\": container with ID starting with a82a81b4742633d5efc04c96622507b7e0321e5056440fc9b0a5e446860cef85 not found: ID does not exist" Jun 13 01:01:55.239303 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239251 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-trusted-ca-bundle\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239303 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239288 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-config-volume\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239412 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239306 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-web-config\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239412 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239332 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwk26\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-kube-api-access-bwk26\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239487 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239450 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-metrics-client-ca\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239487 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239480 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-tls-assets\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239583 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239510 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239583 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239546 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-cluster-tls-config\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239583 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239579 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239728 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239605 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-main-tls\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239728 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239631 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-web\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239728 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239703 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:55.239924 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239733 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-main-db\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.239924 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239736 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:55.239924 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.239773 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-config-out\") pod \"a50a0364-2341-441e-b274-ad9db6ea659a\" (UID: \"a50a0364-2341-441e-b274-ad9db6ea659a\") " Jun 13 01:01:55.240075 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.240058 2552 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-metrics-client-ca\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.240140 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.240078 2552 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.242521 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.242353 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-config-volume" (OuterVolumeSpecName: "config-volume") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:55.242521 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.242473 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-kube-api-access-bwk26" (OuterVolumeSpecName: "kube-api-access-bwk26") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "kube-api-access-bwk26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:01:55.242521 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.242496 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:01:55.242844 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.242806 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:55.243054 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.243036 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:01:55.243146 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.243046 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:55.243239 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.243112 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-config-out" (OuterVolumeSpecName: "config-out") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:01:55.243239 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.243193 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:55.243839 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.243820 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:55.248364 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.248328 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:55.254251 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.254230 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-web-config" (OuterVolumeSpecName: "web-config") pod "a50a0364-2341-441e-b274-ad9db6ea659a" (UID: "a50a0364-2341-441e-b274-ad9db6ea659a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:55.340880 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340849 2552 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-web-config\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.340880 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340878 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwk26\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-kube-api-access-bwk26\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.340880 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340888 2552 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a50a0364-2341-441e-b274-ad9db6ea659a-tls-assets\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.341027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340897 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.341027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340907 2552 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-cluster-tls-config\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.341027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340916 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.341027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340924 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-main-tls\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.341027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340933 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.341027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340943 2552 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-alertmanager-main-db\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.341027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340952 2552 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a50a0364-2341-441e-b274-ad9db6ea659a-config-out\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.341027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.340960 2552 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a50a0364-2341-441e-b274-ad9db6ea659a-config-volume\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:55.405919 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.405895 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 13 01:01:55.411558 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:55.411539 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 13 01:01:56.855183 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:56.855142 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" path="/var/lib/kubelet/pods/a50a0364-2341-441e-b274-ad9db6ea659a/volumes" Jun 13 01:01:58.038728 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.038695 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 13 01:01:58.039305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.039017 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="prometheus" containerID="cri-o://f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" gracePeriod=600 Jun 13 01:01:58.039305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.039072 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy-web" containerID="cri-o://80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" gracePeriod=600 Jun 13 01:01:58.039305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.039092 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy" containerID="cri-o://9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" gracePeriod=600 Jun 13 01:01:58.039305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.039119 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="thanos-sidecar" containerID="cri-o://1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" gracePeriod=600 Jun 13 01:01:58.039305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.039198 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="config-reloader" containerID="cri-o://d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" gracePeriod=600 Jun 13 01:01:58.039583 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.039456 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" gracePeriod=600 Jun 13 01:01:58.285150 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.285125 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:58.370437 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370410 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370618 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370444 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-metrics-client-certs\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370618 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370467 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8gc9\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-kube-api-access-s8gc9\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370618 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370490 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370618 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370560 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-thanos-prometheus-http-client-file\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370618 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370612 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-tls\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370923 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370638 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-metrics-client-ca\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370923 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370662 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config-out\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370923 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370693 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370923 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370722 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-2\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370923 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370753 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-serving-certs-ca-bundle\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370923 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370806 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-db\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370923 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370840 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-tls-assets\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370923 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370866 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-0\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.370923 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370896 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-kubelet-serving-ca-bundle\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.371413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370938 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-1\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.371413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370968 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-kube-rbac-proxy\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.371413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.370997 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-trusted-ca-bundle\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.371413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.371031 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-grpc-tls\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.371413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.371066 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-web-config\") pod \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\" (UID: \"c6129c3f-a643-4ded-95f3-3e91e4fa79f9\") " Jun 13 01:01:58.371413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.371221 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:58.371413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.371352 2552 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.371763 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.371637 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:58.372960 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.372931 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:58.373121 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.373098 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:58.373503 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.373474 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:58.374044 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.374002 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:58.374152 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.374115 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:01:58.374389 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.374352 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 13 01:01:58.374818 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.374795 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config" (OuterVolumeSpecName: "config") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:58.374913 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.374858 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:58.375095 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.375061 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:58.375200 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.375101 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-kube-api-access-s8gc9" (OuterVolumeSpecName: "kube-api-access-s8gc9") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "kube-api-access-s8gc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:01:58.375266 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.375221 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:58.375266 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.375239 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:58.375408 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.375388 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:58.375507 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.375490 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:58.375544 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.375503 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config-out" (OuterVolumeSpecName: "config-out") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:01:58.375749 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.375736 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:01:58.376365 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.376346 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:58.384312 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.384292 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-web-config" (OuterVolumeSpecName: "web-config") pod "c6129c3f-a643-4ded-95f3-3e91e4fa79f9" (UID: "c6129c3f-a643-4ded-95f3-3e91e4fa79f9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:01:58.472538 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472515 2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-db\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472538 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472536 2552 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-tls-assets\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472546 2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472555 2552 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472565 2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472574 2552 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-kube-rbac-proxy\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472583 2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472592 2552 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-grpc-tls\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472600 2552 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-web-config\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472608 2552 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472617 2552 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-metrics-client-certs\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472626 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8gc9\" (UniqueName: \"kubernetes.io/projected/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-kube-api-access-s8gc9\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472634 2552 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472642 2552 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472650 2552 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.472656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472658 2552 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-configmap-metrics-client-ca\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.473139 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472667 2552 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-config-out\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.473139 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472675 2552 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:58.473139 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:58.472684 2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6129c3f-a643-4ded-95f3-3e91e4fa79f9-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:01:59.107760 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107734 2552 generic.go:356] "Generic (PLEG): container finished" podID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerID="5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" exitCode=0 Jun 13 01:01:59.107760 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107758 2552 generic.go:356] "Generic (PLEG): container finished" podID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerID="9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" exitCode=0 Jun 13 01:01:59.107760 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107765 2552 generic.go:356] "Generic (PLEG): container finished" podID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerID="80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" exitCode=0 Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107771 2552 generic.go:356] "Generic (PLEG): container finished" podID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerID="1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" exitCode=0 Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107791 2552 generic.go:356] "Generic (PLEG): container finished" podID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerID="d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" exitCode=0 Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107802 2552 generic.go:356] "Generic (PLEG): container finished" podID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerID="f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" exitCode=0 Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107816 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107813 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerDied","Data":"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a"} Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107947 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerDied","Data":"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea"} Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.107977 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerDied","Data":"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22"} Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.108003 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerDied","Data":"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea"} Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.108023 2552 scope.go:117] "RemoveContainer" containerID="5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.108041 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerDied","Data":"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b"} Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.108080 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerDied","Data":"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718"} Jun 13 01:01:59.108137 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.108118 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6129c3f-a643-4ded-95f3-3e91e4fa79f9","Type":"ContainerDied","Data":"3d3f41f2a2501dac3e0b3a29271f3d883c4c65d64e0729a62778693a11ee8c66"} Jun 13 01:01:59.115492 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.115474 2552 scope.go:117] "RemoveContainer" containerID="9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" Jun 13 01:01:59.122395 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.122379 2552 scope.go:117] "RemoveContainer" containerID="80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" Jun 13 01:01:59.127660 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.127638 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 13 01:01:59.128905 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.128892 2552 scope.go:117] "RemoveContainer" containerID="1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" Jun 13 01:01:59.132561 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.132540 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 13 01:01:59.135449 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.135431 2552 scope.go:117] "RemoveContainer" containerID="d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" Jun 13 01:01:59.141795 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.141764 2552 scope.go:117] "RemoveContainer" containerID="f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" Jun 13 01:01:59.148175 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.148158 2552 scope.go:117] "RemoveContainer" containerID="f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41" Jun 13 01:01:59.154409 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.154383 2552 scope.go:117] "RemoveContainer" containerID="5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" Jun 13 01:01:59.154643 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:59.154624 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": container with ID starting with 5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a not found: ID does not exist" containerID="5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" Jun 13 01:01:59.154727 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.154654 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a"} err="failed to get container status \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": rpc error: code = NotFound desc = could not find container \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": container with ID starting with 5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a not found: ID does not exist" Jun 13 01:01:59.154727 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.154676 2552 scope.go:117] "RemoveContainer" containerID="9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" Jun 13 01:01:59.154954 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:59.154936 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": container with ID starting with 9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea not found: ID does not exist" containerID="9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" Jun 13 01:01:59.154995 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.154960 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea"} err="failed to get container status \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": rpc error: code = NotFound desc = could not find container \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": container with ID starting with 9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea not found: ID does not exist" Jun 13 01:01:59.154995 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.154975 2552 scope.go:117] "RemoveContainer" containerID="80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" Jun 13 01:01:59.155224 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:59.155207 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": container with ID starting with 80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22 not found: ID does not exist" containerID="80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" Jun 13 01:01:59.155286 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.155229 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22"} err="failed to get container status \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": rpc error: code = NotFound desc = could not find container \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": container with ID starting with 80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22 not found: ID does not exist" Jun 13 01:01:59.155286 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.155243 2552 scope.go:117] "RemoveContainer" containerID="1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" Jun 13 01:01:59.155573 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:59.155547 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": container with ID starting with 1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea not found: ID does not exist" containerID="1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" Jun 13 01:01:59.155724 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.155581 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea"} err="failed to get container status \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": rpc error: code = NotFound desc = could not find container \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": container with ID starting with 1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea not found: ID does not exist" Jun 13 01:01:59.155724 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.155601 2552 scope.go:117] "RemoveContainer" containerID="d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" Jun 13 01:01:59.156879 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:59.156857 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": container with ID starting with d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b not found: ID does not exist" containerID="d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" Jun 13 01:01:59.157060 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.156883 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b"} err="failed to get container status \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": rpc error: code = NotFound desc = could not find container \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": container with ID starting with d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b not found: ID does not exist" Jun 13 01:01:59.157060 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.156901 2552 scope.go:117] "RemoveContainer" containerID="f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" Jun 13 01:01:59.157174 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:59.157153 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": container with ID starting with f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718 not found: ID does not exist" containerID="f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" Jun 13 01:01:59.157228 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157172 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718"} err="failed to get container status \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": rpc error: code = NotFound desc = could not find container \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": container with ID starting with f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718 not found: ID does not exist" Jun 13 01:01:59.157228 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157185 2552 scope.go:117] "RemoveContainer" containerID="f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41" Jun 13 01:01:59.157323 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157279 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 13 01:01:59.157423 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:01:59.157406 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": container with ID starting with f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41 not found: ID does not exist" containerID="f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41" Jun 13 01:01:59.157477 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157427 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41"} err="failed to get container status \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": rpc error: code = NotFound desc = could not find container \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": container with ID starting with f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41 not found: ID does not exist" Jun 13 01:01:59.157477 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157440 2552 scope.go:117] "RemoveContainer" containerID="5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" Jun 13 01:01:59.157637 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157622 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="prometheus" Jun 13 01:01:59.157695 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157646 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="prometheus" Jun 13 01:01:59.157695 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157662 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="config-reloader" Jun 13 01:01:59.157695 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157672 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="config-reloader" Jun 13 01:01:59.157695 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157683 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy-metric" Jun 13 01:01:59.157695 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157691 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy-metric" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157627 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a"} err="failed to get container status \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": rpc error: code = NotFound desc = could not find container \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": container with ID starting with 5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a not found: ID does not exist" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157708 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa7355d-fa80-4074-8574-844ade0436f7" containerName="registry" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157712 2552 scope.go:117] "RemoveContainer" containerID="9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157717 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa7355d-fa80-4074-8574-844ade0436f7" containerName="registry" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157730 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="init-config-reloader" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157738 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="init-config-reloader" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157747 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="thanos-sidecar" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157755 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="thanos-sidecar" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157770 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="init-config-reloader" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157793 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="init-config-reloader" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157802 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="config-reloader" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157810 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="config-reloader" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157820 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy-thanos" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157830 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy-thanos" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157843 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="alertmanager" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157852 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="alertmanager" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157861 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy-web" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157869 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy-web" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157878 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157887 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157895 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="prom-label-proxy" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157903 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="prom-label-proxy" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157915 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy-web" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157914 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea"} err="failed to get container status \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": rpc error: code = NotFound desc = could not find container \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": container with ID starting with 9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea not found: ID does not exist" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157924 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy-web" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157927 2552 scope.go:117] "RemoveContainer" containerID="80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157933 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy" Jun 13 01:01:59.157962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.157941 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158018 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy-metric" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158032 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy-web" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158041 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="prometheus" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158051 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="config-reloader" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158059 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="thanos-sidecar" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158067 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158077 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158084 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfa7355d-fa80-4074-8574-844ade0436f7" containerName="registry" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158093 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="config-reloader" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158089 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22"} err="failed to get container status \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": rpc error: code = NotFound desc = could not find container \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": container with ID starting with 80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22 not found: ID does not exist" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158101 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="prom-label-proxy" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158106 2552 scope.go:117] "RemoveContainer" containerID="1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158109 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="alertmanager" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158150 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" containerName="kube-rbac-proxy-thanos" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158159 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a50a0364-2341-441e-b274-ad9db6ea659a" containerName="kube-rbac-proxy-web" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158296 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea"} err="failed to get container status \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": rpc error: code = NotFound desc = could not find container \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": container with ID starting with 1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea not found: ID does not exist" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158312 2552 scope.go:117] "RemoveContainer" containerID="d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158554 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b"} err="failed to get container status \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": rpc error: code = NotFound desc = could not find container \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": container with ID starting with d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b not found: ID does not exist" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158578 2552 scope.go:117] "RemoveContainer" containerID="f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158867 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718"} err="failed to get container status \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": rpc error: code = NotFound desc = could not find container \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": container with ID starting with f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718 not found: ID does not exist" Jun 13 01:01:59.159023 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.158891 2552 scope.go:117] "RemoveContainer" containerID="f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41" Jun 13 01:01:59.159820 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.159143 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41"} err="failed to get container status \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": rpc error: code = NotFound desc = could not find container \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": container with ID starting with f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41 not found: ID does not exist" Jun 13 01:01:59.159820 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.159159 2552 scope.go:117] "RemoveContainer" containerID="5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" Jun 13 01:01:59.159820 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.159393 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a"} err="failed to get container status \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": rpc error: code = NotFound desc = could not find container \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": container with ID starting with 5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a not found: ID does not exist" Jun 13 01:01:59.159820 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.159406 2552 scope.go:117] "RemoveContainer" containerID="9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" Jun 13 01:01:59.159820 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.159604 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea"} err="failed to get container status \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": rpc error: code = NotFound desc = could not find container \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": container with ID starting with 9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea not found: ID does not exist" Jun 13 01:01:59.159820 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.159616 2552 scope.go:117] "RemoveContainer" containerID="80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" Jun 13 01:01:59.160054 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.159919 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22"} err="failed to get container status \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": rpc error: code = NotFound desc = could not find container \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": container with ID starting with 80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22 not found: ID does not exist" Jun 13 01:01:59.160054 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.159945 2552 scope.go:117] "RemoveContainer" containerID="1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" Jun 13 01:01:59.160240 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.160209 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea"} err="failed to get container status \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": rpc error: code = NotFound desc = could not find container \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": container with ID starting with 1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea not found: ID does not exist" Jun 13 01:01:59.160240 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.160230 2552 scope.go:117] "RemoveContainer" containerID="d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" Jun 13 01:01:59.160495 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.160477 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b"} err="failed to get container status \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": rpc error: code = NotFound desc = could not find container \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": container with ID starting with d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b not found: ID does not exist" Jun 13 01:01:59.160547 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.160495 2552 scope.go:117] "RemoveContainer" containerID="f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" Jun 13 01:01:59.160826 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.160760 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718"} err="failed to get container status \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": rpc error: code = NotFound desc = could not find container \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": container with ID starting with f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718 not found: ID does not exist" Jun 13 01:01:59.160884 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.160827 2552 scope.go:117] "RemoveContainer" containerID="f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41" Jun 13 01:01:59.161063 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161043 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41"} err="failed to get container status \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": rpc error: code = NotFound desc = could not find container \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": container with ID starting with f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41 not found: ID does not exist" Jun 13 01:01:59.161063 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161061 2552 scope.go:117] "RemoveContainer" containerID="5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" Jun 13 01:01:59.161279 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161263 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a"} err="failed to get container status \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": rpc error: code = NotFound desc = could not find container \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": container with ID starting with 5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a not found: ID does not exist" Jun 13 01:01:59.161326 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161280 2552 scope.go:117] "RemoveContainer" containerID="9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" Jun 13 01:01:59.161505 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161487 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea"} err="failed to get container status \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": rpc error: code = NotFound desc = could not find container \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": container with ID starting with 9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea not found: ID does not exist" Jun 13 01:01:59.161575 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161510 2552 scope.go:117] "RemoveContainer" containerID="80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" Jun 13 01:01:59.161737 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161713 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22"} err="failed to get container status \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": rpc error: code = NotFound desc = could not find container \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": container with ID starting with 80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22 not found: ID does not exist" Jun 13 01:01:59.161804 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161739 2552 scope.go:117] "RemoveContainer" containerID="1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" Jun 13 01:01:59.161960 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161945 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.162000 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161963 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea"} err="failed to get container status \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": rpc error: code = NotFound desc = could not find container \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": container with ID starting with 1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea not found: ID does not exist" Jun 13 01:01:59.162000 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.161979 2552 scope.go:117] "RemoveContainer" containerID="d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" Jun 13 01:01:59.162260 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.162233 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b"} err="failed to get container status \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": rpc error: code = NotFound desc = could not find container \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": container with ID starting with d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b not found: ID does not exist" Jun 13 01:01:59.162332 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.162261 2552 scope.go:117] "RemoveContainer" containerID="f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" Jun 13 01:01:59.162612 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.162590 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718"} err="failed to get container status \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": rpc error: code = NotFound desc = could not find container \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": container with ID starting with f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718 not found: ID does not exist" Jun 13 01:01:59.162671 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.162616 2552 scope.go:117] "RemoveContainer" containerID="f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41" Jun 13 01:01:59.162877 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.162858 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41"} err="failed to get container status \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": rpc error: code = NotFound desc = could not find container \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": container with ID starting with f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41 not found: ID does not exist" Jun 13 01:01:59.162877 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.162877 2552 scope.go:117] "RemoveContainer" containerID="5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" Jun 13 01:01:59.163133 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.163113 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a"} err="failed to get container status \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": rpc error: code = NotFound desc = could not find container \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": container with ID starting with 5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a not found: ID does not exist" Jun 13 01:01:59.163181 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.163137 2552 scope.go:117] "RemoveContainer" containerID="9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" Jun 13 01:01:59.163374 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.163355 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea"} err="failed to get container status \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": rpc error: code = NotFound desc = could not find container \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": container with ID starting with 9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea not found: ID does not exist" Jun 13 01:01:59.163440 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.163374 2552 scope.go:117] "RemoveContainer" containerID="80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" Jun 13 01:01:59.163554 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.163534 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22"} err="failed to get container status \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": rpc error: code = NotFound desc = could not find container \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": container with ID starting with 80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22 not found: ID does not exist" Jun 13 01:01:59.163595 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.163559 2552 scope.go:117] "RemoveContainer" containerID="1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" Jun 13 01:01:59.163801 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.163761 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea"} err="failed to get container status \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": rpc error: code = NotFound desc = could not find container \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": container with ID starting with 1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea not found: ID does not exist" Jun 13 01:01:59.163801 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.163792 2552 scope.go:117] "RemoveContainer" containerID="d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" Jun 13 01:01:59.164031 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164016 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b"} err="failed to get container status \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": rpc error: code = NotFound desc = could not find container \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": container with ID starting with d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b not found: ID does not exist" Jun 13 01:01:59.164088 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164030 2552 scope.go:117] "RemoveContainer" containerID="f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" Jun 13 01:01:59.164283 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164263 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718"} err="failed to get container status \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": rpc error: code = NotFound desc = could not find container \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": container with ID starting with f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718 not found: ID does not exist" Jun 13 01:01:59.164283 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164283 2552 scope.go:117] "RemoveContainer" containerID="f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41" Jun 13 01:01:59.164398 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164302 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 13 01:01:59.164398 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164345 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4l7xx\"" Jun 13 01:01:59.164483 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164453 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 13 01:01:59.164558 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164537 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41"} err="failed to get container status \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": rpc error: code = NotFound desc = could not find container \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": container with ID starting with f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41 not found: ID does not exist" Jun 13 01:01:59.164558 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164558 2552 scope.go:117] "RemoveContainer" containerID="5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a" Jun 13 01:01:59.164663 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164641 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-agog0fucjmlfu\"" Jun 13 01:01:59.164663 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164647 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 13 01:01:59.164825 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164803 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a"} err="failed to get container status \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": rpc error: code = NotFound desc = could not find container \"5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a\": container with ID starting with 5afc12d7e608e4e6216c9b1e923a2798f1666d6a973f09a091daaf0b6da5b76a not found: ID does not exist" Jun 13 01:01:59.164885 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164829 2552 scope.go:117] "RemoveContainer" containerID="9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea" Jun 13 01:01:59.164939 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164912 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 13 01:01:59.164990 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.164942 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 13 01:01:59.165083 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165066 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 13 01:01:59.165083 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165074 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 13 01:01:59.165183 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165141 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 13 01:01:59.165235 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165182 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea"} err="failed to get container status \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": rpc error: code = NotFound desc = could not find container \"9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea\": container with ID starting with 9f78cdc98a22a051c5926dca148a457cfb83d28b3fb0851964ff650690faabea not found: ID does not exist" Jun 13 01:01:59.165235 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165214 2552 scope.go:117] "RemoveContainer" containerID="80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22" Jun 13 01:01:59.165387 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165368 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 13 01:01:59.165497 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165439 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22"} err="failed to get container status \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": rpc error: code = NotFound desc = could not find container \"80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22\": container with ID starting with 80a4909f4651796759ecbefd81f570cd7fda4e72d25088c294ea07aa98c35c22 not found: ID does not exist" Jun 13 01:01:59.165497 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165473 2552 scope.go:117] "RemoveContainer" containerID="1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea" Jun 13 01:01:59.165663 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165474 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 13 01:01:59.165663 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165559 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 13 01:01:59.165663 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165637 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 13 01:01:59.165857 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165673 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea"} err="failed to get container status \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": rpc error: code = NotFound desc = could not find container \"1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea\": container with ID starting with 1b35afbcbfde2d820fdd59e8fe07ccce251a78dd5090e51dc5e3e49441b756ea not found: ID does not exist" Jun 13 01:01:59.165857 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165693 2552 scope.go:117] "RemoveContainer" containerID="d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b" Jun 13 01:01:59.165932 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165861 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 13 01:01:59.165983 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165936 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b"} err="failed to get container status \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": rpc error: code = NotFound desc = could not find container \"d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b\": container with ID starting with d39cf6239f806e66776f0d3259115282a9814d319fec2510dfa78f575847036b not found: ID does not exist" Jun 13 01:01:59.165983 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.165958 2552 scope.go:117] "RemoveContainer" containerID="f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718" Jun 13 01:01:59.166272 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.166232 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718"} err="failed to get container status \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": rpc error: code = NotFound desc = could not find container \"f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718\": container with ID starting with f6b823544ef9c771131b4031f2671e476dedc907892dee722b3052a2c0555718 not found: ID does not exist" Jun 13 01:01:59.166272 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.166256 2552 scope.go:117] "RemoveContainer" containerID="f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41" Jun 13 01:01:59.166698 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.166673 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41"} err="failed to get container status \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": rpc error: code = NotFound desc = could not find container \"f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41\": container with ID starting with f82be3cb41dd290166b9a0844ff57997b04340c5d64a45702347de72456a8d41 not found: ID does not exist" Jun 13 01:01:59.168135 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.167665 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 13 01:01:59.171334 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.171314 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 13 01:01:59.177839 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.177811 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.177950 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.177847 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.177950 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.177874 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.177950 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.177915 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt8bc\" (UniqueName: \"kubernetes.io/projected/71cad2aa-4868-4997-8a27-caa57306c0e4-kube-api-access-bt8bc\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178101 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.177978 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71cad2aa-4868-4997-8a27-caa57306c0e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178101 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178006 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178101 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178044 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178101 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178075 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178268 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178131 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178268 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178158 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178268 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178191 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178268 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178203 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 13 01:01:59.178268 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178216 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178268 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178240 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-config\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178268 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178267 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178577 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178293 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178577 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178310 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178577 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178357 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178577 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178408 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178577 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178442 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71cad2aa-4868-4997-8a27-caa57306c0e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.178577 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.178465 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279041 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279011 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71cad2aa-4868-4997-8a27-caa57306c0e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279138 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279043 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279138 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279068 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279246 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279210 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279302 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279268 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279353 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279297 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt8bc\" (UniqueName: \"kubernetes.io/projected/71cad2aa-4868-4997-8a27-caa57306c0e4-kube-api-access-bt8bc\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279353 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279327 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71cad2aa-4868-4997-8a27-caa57306c0e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279456 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279356 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279456 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279389 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279456 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279425 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279633 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279486 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279633 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279516 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279633 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279543 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279633 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279571 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279633 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279598 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-config\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.279633 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279626 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.280125 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279656 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.280125 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279680 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.280125 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279708 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.280125 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279735 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.280125 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.279918 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.280125 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.280082 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.282164 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.280133 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.282164 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.280348 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.282164 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.280949 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.282164 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.281009 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.282164 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.281897 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.283212 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.282794 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71cad2aa-4868-4997-8a27-caa57306c0e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.283212 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.283048 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-config\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.283212 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.283204 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71cad2aa-4868-4997-8a27-caa57306c0e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.283657 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.283619 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.283738 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.283689 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.284428 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.284385 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.284589 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.284546 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71cad2aa-4868-4997-8a27-caa57306c0e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.284909 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.284891 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.284987 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.284969 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.285441 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.285421 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.285518 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.285476 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.286476 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.286450 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71cad2aa-4868-4997-8a27-caa57306c0e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.287823 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.287759 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt8bc\" (UniqueName: \"kubernetes.io/projected/71cad2aa-4868-4997-8a27-caa57306c0e4-kube-api-access-bt8bc\") pod \"prometheus-k8s-0\" (UID: \"71cad2aa-4868-4997-8a27-caa57306c0e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.474530 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.474459 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:01:59.606216 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:01:59.606030 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 13 01:01:59.610283 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:01:59.610256 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71cad2aa_4868_4997_8a27_caa57306c0e4.slice/crio-bc6dd7837f8dc0506b2725d727c33236a8703a5a10cb0fdd47cb51b9a910f5da WatchSource:0}: Error finding container bc6dd7837f8dc0506b2725d727c33236a8703a5a10cb0fdd47cb51b9a910f5da: Status 404 returned error can't find the container with id bc6dd7837f8dc0506b2725d727c33236a8703a5a10cb0fdd47cb51b9a910f5da Jun 13 01:02:00.113505 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:00.113472 2552 generic.go:356] "Generic (PLEG): container finished" podID="71cad2aa-4868-4997-8a27-caa57306c0e4" containerID="d119e4e910748f267c68ac4f3cceeb7958fbccff15337cdf40bd6008ecf9ca7e" exitCode=0 Jun 13 01:02:00.113852 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:00.113564 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71cad2aa-4868-4997-8a27-caa57306c0e4","Type":"ContainerDied","Data":"d119e4e910748f267c68ac4f3cceeb7958fbccff15337cdf40bd6008ecf9ca7e"} Jun 13 01:02:00.113852 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:00.113650 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71cad2aa-4868-4997-8a27-caa57306c0e4","Type":"ContainerStarted","Data":"bc6dd7837f8dc0506b2725d727c33236a8703a5a10cb0fdd47cb51b9a910f5da"} Jun 13 01:02:00.855737 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:00.855710 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6129c3f-a643-4ded-95f3-3e91e4fa79f9" path="/var/lib/kubelet/pods/c6129c3f-a643-4ded-95f3-3e91e4fa79f9/volumes" Jun 13 01:02:01.121285 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:01.121228 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71cad2aa-4868-4997-8a27-caa57306c0e4","Type":"ContainerStarted","Data":"8e34f431155b941a91875c77a5758d49ae0a4834033e9c36727c4d297946c99d"} Jun 13 01:02:01.121285 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:01.121281 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71cad2aa-4868-4997-8a27-caa57306c0e4","Type":"ContainerStarted","Data":"ea77d7c197df9191c9d9c9e220d4aa55d2713130ba5f3be6f1cdb25d0a7066dd"} Jun 13 01:02:01.121595 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:01.121308 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71cad2aa-4868-4997-8a27-caa57306c0e4","Type":"ContainerStarted","Data":"eb2562b4920cad17db2b03d439c06686dd8fd168aad4142e10f8c0988fab77de"} Jun 13 01:02:01.121595 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:01.121333 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71cad2aa-4868-4997-8a27-caa57306c0e4","Type":"ContainerStarted","Data":"6b6ff6fc77913711b7d58412cdbdfe132fb1f84c3c55f69fdace6ab19ee0e9f0"} Jun 13 01:02:01.121595 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:01.121361 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71cad2aa-4868-4997-8a27-caa57306c0e4","Type":"ContainerStarted","Data":"3107e20f119c663a371dafc1ad5427523a305b1dc3da3371974d56f22dc9fc96"} Jun 13 01:02:01.121595 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:01.121383 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71cad2aa-4868-4997-8a27-caa57306c0e4","Type":"ContainerStarted","Data":"26f3f2299675d723e9dab9e4e1ab660c6e45f52ac70c6bd77630f1e3b5d9b8fe"} Jun 13 01:02:01.148455 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:01.148409 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.1289936799999998 podStartE2EDuration="2.148318189s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.324509ms" isStatefulPod=false podCreationTimestamp="2026-06-13 01:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:02:01.146204775 +0000 UTC m=+360.720133691" watchObservedRunningTime="2026-06-13 01:02:01.148318189 +0000 UTC m=+360.722247077" Jun 13 01:02:04.475368 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:04.475331 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:02:14.702305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:14.702258 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 01:02:14.704499 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:14.704477 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ae3796c-d916-4f7d-ac66-24899ca4e0f0-metrics-certs\") pod \"network-metrics-daemon-ln4ft\" (UID: \"7ae3796c-d916-4f7d-ac66-24899ca4e0f0\") " pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 01:02:14.953224 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:14.953160 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m8nk7\"" Jun 13 01:02:14.960957 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:14.960939 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ln4ft" Jun 13 01:02:15.084843 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:15.084733 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ln4ft"] Jun 13 01:02:15.087515 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:02:15.087488 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae3796c_d916_4f7d_ac66_24899ca4e0f0.slice/crio-9de7e17f169aff36b64a6a6bb345598cf58b2f4f32b74c2440779e1271194cd8 WatchSource:0}: Error finding container 9de7e17f169aff36b64a6a6bb345598cf58b2f4f32b74c2440779e1271194cd8: Status 404 returned error can't find the container with id 9de7e17f169aff36b64a6a6bb345598cf58b2f4f32b74c2440779e1271194cd8 Jun 13 01:02:15.089919 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:15.089903 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 13 01:02:15.180413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:15.180383 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ln4ft" event={"ID":"7ae3796c-d916-4f7d-ac66-24899ca4e0f0","Type":"ContainerStarted","Data":"9de7e17f169aff36b64a6a6bb345598cf58b2f4f32b74c2440779e1271194cd8"} Jun 13 01:02:16.186165 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:16.186090 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ln4ft" event={"ID":"7ae3796c-d916-4f7d-ac66-24899ca4e0f0","Type":"ContainerStarted","Data":"94416af6ea5ed216a5920d6e156505ccf0a2ef69bdc7d22b5efe701714e6ad26"} Jun 13 01:02:16.186165 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:16.186146 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ln4ft" event={"ID":"7ae3796c-d916-4f7d-ac66-24899ca4e0f0","Type":"ContainerStarted","Data":"4b50c3ba8fa13c8c6d49389dd339a6891662d970c084b1cbd997f2f24b780785"} Jun 13 01:02:16.201530 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:16.201488 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ln4ft" podStartSLOduration=374.395803752 podStartE2EDuration="6m15.201473476s" totalImagesPullingTime="805.669724ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 00:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:02:16.200078534 +0000 UTC m=+375.774007422" watchObservedRunningTime="2026-06-13 01:02:16.201473476 +0000 UTC m=+375.775402413" Jun 13 01:02:59.474698 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:59.474658 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:02:59.490127 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:02:59.490105 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:03:00.368263 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:03:00.368237 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 13 01:04:11.420475 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.420435 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-qbknk"] Jun 13 01:04:11.423899 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.423875 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-qbknk" Jun 13 01:04:11.426571 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.426549 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-n6xpd\"" Jun 13 01:04:11.426571 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.426568 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 13 01:04:11.427409 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.427383 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 13 01:04:11.434327 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.434300 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-qbknk"] Jun 13 01:04:11.559607 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.559583 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1393ec2b-b06c-413a-ab76-02c25fed24d9-bound-sa-token\") pod \"cert-manager-59c77684cc-qbknk\" (UID: \"1393ec2b-b06c-413a-ab76-02c25fed24d9\") " pod="cert-manager/cert-manager-59c77684cc-qbknk" Jun 13 01:04:11.559699 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.559622 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpdvd\" (UniqueName: \"kubernetes.io/projected/1393ec2b-b06c-413a-ab76-02c25fed24d9-kube-api-access-dpdvd\") pod \"cert-manager-59c77684cc-qbknk\" (UID: \"1393ec2b-b06c-413a-ab76-02c25fed24d9\") " pod="cert-manager/cert-manager-59c77684cc-qbknk" Jun 13 01:04:11.660126 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.660099 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpdvd\" (UniqueName: \"kubernetes.io/projected/1393ec2b-b06c-413a-ab76-02c25fed24d9-kube-api-access-dpdvd\") pod \"cert-manager-59c77684cc-qbknk\" (UID: \"1393ec2b-b06c-413a-ab76-02c25fed24d9\") " pod="cert-manager/cert-manager-59c77684cc-qbknk" Jun 13 01:04:11.660243 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.660153 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1393ec2b-b06c-413a-ab76-02c25fed24d9-bound-sa-token\") pod \"cert-manager-59c77684cc-qbknk\" (UID: \"1393ec2b-b06c-413a-ab76-02c25fed24d9\") " pod="cert-manager/cert-manager-59c77684cc-qbknk" Jun 13 01:04:11.667726 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.667701 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1393ec2b-b06c-413a-ab76-02c25fed24d9-bound-sa-token\") pod \"cert-manager-59c77684cc-qbknk\" (UID: \"1393ec2b-b06c-413a-ab76-02c25fed24d9\") " pod="cert-manager/cert-manager-59c77684cc-qbknk" Jun 13 01:04:11.667831 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.667812 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpdvd\" (UniqueName: \"kubernetes.io/projected/1393ec2b-b06c-413a-ab76-02c25fed24d9-kube-api-access-dpdvd\") pod \"cert-manager-59c77684cc-qbknk\" (UID: \"1393ec2b-b06c-413a-ab76-02c25fed24d9\") " pod="cert-manager/cert-manager-59c77684cc-qbknk" Jun 13 01:04:11.742874 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.742824 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-qbknk" Jun 13 01:04:11.868178 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:11.868152 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-qbknk"] Jun 13 01:04:11.871667 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:04:11.871643 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1393ec2b_b06c_413a_ab76_02c25fed24d9.slice/crio-a143a5c934b5242b19bf9e67ec6e997f0c15dfbf96b5c9b29c16bd9e30b56692 WatchSource:0}: Error finding container a143a5c934b5242b19bf9e67ec6e997f0c15dfbf96b5c9b29c16bd9e30b56692: Status 404 returned error can't find the container with id a143a5c934b5242b19bf9e67ec6e997f0c15dfbf96b5c9b29c16bd9e30b56692 Jun 13 01:04:12.629203 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:12.629152 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-qbknk" event={"ID":"1393ec2b-b06c-413a-ab76-02c25fed24d9","Type":"ContainerStarted","Data":"a143a5c934b5242b19bf9e67ec6e997f0c15dfbf96b5c9b29c16bd9e30b56692"} Jun 13 01:04:14.639136 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:14.639103 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-qbknk" event={"ID":"1393ec2b-b06c-413a-ab76-02c25fed24d9","Type":"ContainerStarted","Data":"b003aa805bd49a8444c623611d546c41c732acdf186785d5173cd75acd282e64"} Jun 13 01:04:14.655140 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:14.655099 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-qbknk" podStartSLOduration=0.974448779 podStartE2EDuration="3.655083453s" totalImagesPullingTime="2.680634674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:04:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:04:14.6532932 +0000 UTC m=+494.227222089" watchObservedRunningTime="2026-06-13 01:04:14.655083453 +0000 UTC m=+494.229012341" Jun 13 01:04:18.987081 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:18.987047 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp"] Jun 13 01:04:18.990522 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:18.990499 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:18.993015 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:18.992989 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 13 01:04:18.993269 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:18.993249 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 13 01:04:18.993345 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:18.993248 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-pjwnh\"" Jun 13 01:04:18.994144 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:18.994119 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 13 01:04:18.994247 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:18.994121 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 13 01:04:18.994247 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:18.994164 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 13 01:04:19.001406 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.001385 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp"] Jun 13 01:04:19.117330 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.117306 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/94606d7c-075f-4b0a-abdf-5adb1ccb6823-manager-config\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.117433 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.117340 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/94606d7c-075f-4b0a-abdf-5adb1ccb6823-metrics-cert\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.117507 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.117439 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94606d7c-075f-4b0a-abdf-5adb1ccb6823-cert\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.117507 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.117486 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvp7w\" (UniqueName: \"kubernetes.io/projected/94606d7c-075f-4b0a-abdf-5adb1ccb6823-kube-api-access-dvp7w\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.218661 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.218636 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94606d7c-075f-4b0a-abdf-5adb1ccb6823-cert\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.218767 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.218671 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvp7w\" (UniqueName: \"kubernetes.io/projected/94606d7c-075f-4b0a-abdf-5adb1ccb6823-kube-api-access-dvp7w\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.218767 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.218701 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/94606d7c-075f-4b0a-abdf-5adb1ccb6823-manager-config\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.218767 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.218724 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/94606d7c-075f-4b0a-abdf-5adb1ccb6823-metrics-cert\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.219319 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.219295 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/94606d7c-075f-4b0a-abdf-5adb1ccb6823-manager-config\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.221303 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.221281 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/94606d7c-075f-4b0a-abdf-5adb1ccb6823-metrics-cert\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.221382 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.221331 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94606d7c-075f-4b0a-abdf-5adb1ccb6823-cert\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.229396 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.229375 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvp7w\" (UniqueName: \"kubernetes.io/projected/94606d7c-075f-4b0a-abdf-5adb1ccb6823-kube-api-access-dvp7w\") pod \"lws-controller-manager-6dc4584f75-nbvpp\" (UID: \"94606d7c-075f-4b0a-abdf-5adb1ccb6823\") " pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.300364 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.300344 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:19.427510 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.427482 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp"] Jun 13 01:04:19.430395 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:04:19.430370 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94606d7c_075f_4b0a_abdf_5adb1ccb6823.slice/crio-3abd28ea1694487669572c65d5a035d62f2a20effe8c3c3c194c127805a0d159 WatchSource:0}: Error finding container 3abd28ea1694487669572c65d5a035d62f2a20effe8c3c3c194c127805a0d159: Status 404 returned error can't find the container with id 3abd28ea1694487669572c65d5a035d62f2a20effe8c3c3c194c127805a0d159 Jun 13 01:04:19.659424 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:19.659351 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" event={"ID":"94606d7c-075f-4b0a-abdf-5adb1ccb6823","Type":"ContainerStarted","Data":"3abd28ea1694487669572c65d5a035d62f2a20effe8c3c3c194c127805a0d159"} Jun 13 01:04:21.668884 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:21.668846 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" event={"ID":"94606d7c-075f-4b0a-abdf-5adb1ccb6823","Type":"ContainerStarted","Data":"1bf32c99a0b6ce154eee4dd9b1d32205c57d077b8bbbeb71b8bef51d8e98533c"} Jun 13 01:04:21.669197 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:21.668929 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:21.685642 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:21.685596 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" podStartSLOduration=1.570945984 podStartE2EDuration="3.685581238s" totalImagesPullingTime="2.114635254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:04:21.684618412 +0000 UTC m=+501.258547301" watchObservedRunningTime="2026-06-13 01:04:21.685581238 +0000 UTC m=+501.259510126" Jun 13 01:04:24.189451 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.189409 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5"] Jun 13 01:04:24.193020 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.192997 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.198212 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.198190 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 13 01:04:24.198539 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.198510 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7kh2m\"" Jun 13 01:04:24.198709 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.198689 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 13 01:04:24.199094 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.199070 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 13 01:04:24.200615 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.200591 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 13 01:04:24.214113 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.214091 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5"] Jun 13 01:04:24.358913 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.358888 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e26902e2-7b84-4a5d-a93a-de08abe4de05-tmp\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.359027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.358930 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md9mh\" (UniqueName: \"kubernetes.io/projected/e26902e2-7b84-4a5d-a93a-de08abe4de05-kube-api-access-md9mh\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.359027 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.358967 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e26902e2-7b84-4a5d-a93a-de08abe4de05-manifests\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.359145 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.359032 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e26902e2-7b84-4a5d-a93a-de08abe4de05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.359145 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.359088 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e26902e2-7b84-4a5d-a93a-de08abe4de05-webhook-cert\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.459897 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.459841 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md9mh\" (UniqueName: \"kubernetes.io/projected/e26902e2-7b84-4a5d-a93a-de08abe4de05-kube-api-access-md9mh\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.459897 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.459889 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e26902e2-7b84-4a5d-a93a-de08abe4de05-manifests\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.460062 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.459913 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e26902e2-7b84-4a5d-a93a-de08abe4de05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.460062 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.459935 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e26902e2-7b84-4a5d-a93a-de08abe4de05-webhook-cert\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.460062 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.460046 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e26902e2-7b84-4a5d-a93a-de08abe4de05-tmp\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.460303 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.460281 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e26902e2-7b84-4a5d-a93a-de08abe4de05-manifests\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.462248 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.462219 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e26902e2-7b84-4a5d-a93a-de08abe4de05-tmp\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.462558 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.462539 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e26902e2-7b84-4a5d-a93a-de08abe4de05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.462602 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.462573 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e26902e2-7b84-4a5d-a93a-de08abe4de05-webhook-cert\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.475025 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.472136 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md9mh\" (UniqueName: \"kubernetes.io/projected/e26902e2-7b84-4a5d-a93a-de08abe4de05-kube-api-access-md9mh\") pod \"opendatahub-operator-controller-manager-78dd6f457b-g7nx5\" (UID: \"e26902e2-7b84-4a5d-a93a-de08abe4de05\") " pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.503025 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.503000 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:24.669529 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.669498 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5"] Jun 13 01:04:24.673334 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:04:24.673291 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26902e2_7b84_4a5d_a93a_de08abe4de05.slice/crio-8c18c4501ef05b61e301406dfbeefc2d8d555ffe359a6b361e8cdc9a123e7663 WatchSource:0}: Error finding container 8c18c4501ef05b61e301406dfbeefc2d8d555ffe359a6b361e8cdc9a123e7663: Status 404 returned error can't find the container with id 8c18c4501ef05b61e301406dfbeefc2d8d555ffe359a6b361e8cdc9a123e7663 Jun 13 01:04:24.682529 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:24.682495 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" event={"ID":"e26902e2-7b84-4a5d-a93a-de08abe4de05","Type":"ContainerStarted","Data":"8c18c4501ef05b61e301406dfbeefc2d8d555ffe359a6b361e8cdc9a123e7663"} Jun 13 01:04:27.699014 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:27.698941 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" event={"ID":"e26902e2-7b84-4a5d-a93a-de08abe4de05","Type":"ContainerStarted","Data":"f7e9bbc6e2c4859e652d585c75f3e80412d76c7a99daa7c4d2eaeadd7dcb24b4"} Jun 13 01:04:28.710284 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:28.710213 2552 generic.go:356] "Generic (PLEG): container finished" podID="e26902e2-7b84-4a5d-a93a-de08abe4de05" containerID="f7e9bbc6e2c4859e652d585c75f3e80412d76c7a99daa7c4d2eaeadd7dcb24b4" exitCode=0 Jun 13 01:04:28.710621 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:28.710294 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" event={"ID":"e26902e2-7b84-4a5d-a93a-de08abe4de05","Type":"ContainerDied","Data":"f7e9bbc6e2c4859e652d585c75f3e80412d76c7a99daa7c4d2eaeadd7dcb24b4"} Jun 13 01:04:29.716608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:29.716575 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" event={"ID":"e26902e2-7b84-4a5d-a93a-de08abe4de05","Type":"ContainerStarted","Data":"4d1c8b49761d6d7faf6d152cbf6f96bc0204b606fe5fc0b6aad48eeca092a5a8"} Jun 13 01:04:29.716974 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:29.716673 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:04:29.742540 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:29.742498 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" podStartSLOduration=1.747885949 podStartE2EDuration="5.742485603s" totalImagesPullingTime="3.017824666s" totalInitContainerRuntime="976.774988ms" isStatefulPod=false podCreationTimestamp="2026-06-13 01:04:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:04:29.740266807 +0000 UTC m=+509.314195729" watchObservedRunningTime="2026-06-13 01:04:29.742485603 +0000 UTC m=+509.316414490" Jun 13 01:04:32.675580 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:32.675549 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6dc4584f75-nbvpp" Jun 13 01:04:40.722628 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:04:40.722597 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78dd6f457b-g7nx5" Jun 13 01:05:11.452618 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.452578 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8"] Jun 13 01:05:11.462940 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.462917 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.466004 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.465975 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fhklh\"" Jun 13 01:05:11.466004 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.465982 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 13 01:05:11.466259 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.466235 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8"] Jun 13 01:05:11.616196 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.616173 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.616294 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.616201 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.616294 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.616218 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.616294 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.616240 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.616398 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.616286 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6nvz\" (UniqueName: \"kubernetes.io/projected/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-kube-api-access-s6nvz\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.616398 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.616359 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.616398 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.616384 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.616525 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.616412 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.616525 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.616477 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.716903 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.716831 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.716903 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.716878 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717059 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.716996 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717059 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717030 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717059 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717045 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717208 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717074 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717208 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717120 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6nvz\" (UniqueName: \"kubernetes.io/projected/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-kube-api-access-s6nvz\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717208 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717163 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717208 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717201 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717384 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717227 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717520 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717490 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717650 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717521 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717650 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717570 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.717831 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.717811 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.719209 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.719190 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.719508 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.719481 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.727429 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.727411 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.730238 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.730216 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6nvz\" (UniqueName: \"kubernetes.io/projected/c986a91d-2ddd-4f79-91d4-9e86a7a475f8-kube-api-access-s6nvz\") pod \"data-science-gateway-data-science-gateway-class-5b556588bc9tgn8\" (UID: \"c986a91d-2ddd-4f79-91d4-9e86a7a475f8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.774577 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.774559 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:11.913149 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:11.913120 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8"] Jun 13 01:05:11.916872 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:05:11.916840 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc986a91d_2ddd_4f79_91d4_9e86a7a475f8.slice/crio-f062ef2b101bffbbe6c4bbe864326d70b92f34a16f208d9d8d4c42e84c520530 WatchSource:0}: Error finding container f062ef2b101bffbbe6c4bbe864326d70b92f34a16f208d9d8d4c42e84c520530: Status 404 returned error can't find the container with id f062ef2b101bffbbe6c4bbe864326d70b92f34a16f208d9d8d4c42e84c520530 Jun 13 01:05:12.909028 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:12.908985 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" event={"ID":"c986a91d-2ddd-4f79-91d4-9e86a7a475f8","Type":"ContainerStarted","Data":"f062ef2b101bffbbe6c4bbe864326d70b92f34a16f208d9d8d4c42e84c520530"} Jun 13 01:05:14.210072 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:14.210033 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 13 01:05:14.210405 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:14.210127 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 13 01:05:14.210405 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:14.210156 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 13 01:05:14.919068 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:14.919035 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" event={"ID":"c986a91d-2ddd-4f79-91d4-9e86a7a475f8","Type":"ContainerStarted","Data":"248eea173a23d1bcf2440d97f74de2b60d494e94d8dc4db08c3d9ad5c6890eec"} Jun 13 01:05:14.940512 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:14.940471 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" podStartSLOduration=1.649500889 podStartE2EDuration="3.940457804s" totalImagesPullingTime="2.290956915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:05:14.939153421 +0000 UTC m=+554.513082337" watchObservedRunningTime="2026-06-13 01:05:14.940457804 +0000 UTC m=+554.514386691" Jun 13 01:05:15.306841 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.306807 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-86dc5dcf99-z59ht"] Jun 13 01:05:15.309515 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.309496 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" Jun 13 01:05:15.311920 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.311894 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 13 01:05:15.312016 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.311933 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5g94w\"" Jun 13 01:05:15.318131 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.318109 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-86dc5dcf99-z59ht"] Jun 13 01:05:15.447471 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.447442 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hb8r\" (UniqueName: \"kubernetes.io/projected/2ec3aeec-0939-41bc-86f7-05e36d05951f-kube-api-access-2hb8r\") pod \"dashboard-redirect-86dc5dcf99-z59ht\" (UID: \"2ec3aeec-0939-41bc-86f7-05e36d05951f\") " pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" Jun 13 01:05:15.447623 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.447531 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/2ec3aeec-0939-41bc-86f7-05e36d05951f-redirect-config\") pod \"dashboard-redirect-86dc5dcf99-z59ht\" (UID: \"2ec3aeec-0939-41bc-86f7-05e36d05951f\") " pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" Jun 13 01:05:15.548506 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.548462 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/2ec3aeec-0939-41bc-86f7-05e36d05951f-redirect-config\") pod \"dashboard-redirect-86dc5dcf99-z59ht\" (UID: \"2ec3aeec-0939-41bc-86f7-05e36d05951f\") " pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" Jun 13 01:05:15.548679 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.548534 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hb8r\" (UniqueName: \"kubernetes.io/projected/2ec3aeec-0939-41bc-86f7-05e36d05951f-kube-api-access-2hb8r\") pod \"dashboard-redirect-86dc5dcf99-z59ht\" (UID: \"2ec3aeec-0939-41bc-86f7-05e36d05951f\") " pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" Jun 13 01:05:15.549148 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.549127 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/2ec3aeec-0939-41bc-86f7-05e36d05951f-redirect-config\") pod \"dashboard-redirect-86dc5dcf99-z59ht\" (UID: \"2ec3aeec-0939-41bc-86f7-05e36d05951f\") " pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" Jun 13 01:05:15.556078 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.556051 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hb8r\" (UniqueName: \"kubernetes.io/projected/2ec3aeec-0939-41bc-86f7-05e36d05951f-kube-api-access-2hb8r\") pod \"dashboard-redirect-86dc5dcf99-z59ht\" (UID: \"2ec3aeec-0939-41bc-86f7-05e36d05951f\") " pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" Jun 13 01:05:15.622038 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.621986 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" Jun 13 01:05:15.764022 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.763993 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-86dc5dcf99-z59ht"] Jun 13 01:05:15.767430 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:05:15.767401 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ec3aeec_0939_41bc_86f7_05e36d05951f.slice/crio-451902dce53a0a1a7b3d306444f96eb67838f4cdf9e5b44dcdca3f8be64b9085 WatchSource:0}: Error finding container 451902dce53a0a1a7b3d306444f96eb67838f4cdf9e5b44dcdca3f8be64b9085: Status 404 returned error can't find the container with id 451902dce53a0a1a7b3d306444f96eb67838f4cdf9e5b44dcdca3f8be64b9085 Jun 13 01:05:15.775367 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.775344 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:15.779653 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.779636 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:15.924085 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.924024 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" event={"ID":"2ec3aeec-0939-41bc-86f7-05e36d05951f","Type":"ContainerStarted","Data":"451902dce53a0a1a7b3d306444f96eb67838f4cdf9e5b44dcdca3f8be64b9085"} Jun 13 01:05:15.924222 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.924205 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:15.924803 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:15.924772 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bc9tgn8" Jun 13 01:05:17.936866 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:17.936830 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" event={"ID":"2ec3aeec-0939-41bc-86f7-05e36d05951f","Type":"ContainerStarted","Data":"2c07dcfa393934070158d42604d3fbca49d9135e5cd70afceb32b9b97ebb7a5e"} Jun 13 01:05:17.979663 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:17.979616 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-86dc5dcf99-z59ht" podStartSLOduration=1.330781636 podStartE2EDuration="2.979601565s" totalImagesPullingTime="1.648819929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:05:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:05:17.979113694 +0000 UTC m=+557.553042580" watchObservedRunningTime="2026-06-13 01:05:17.979601565 +0000 UTC m=+557.553530451" Jun 13 01:05:44.555572 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.555526 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8wcdx"] Jun 13 01:05:44.558121 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.558092 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" Jun 13 01:05:44.560550 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.560523 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 13 01:05:44.560550 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.560544 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 13 01:05:44.560716 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.560586 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-4qbtg\"" Jun 13 01:05:44.567577 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.567496 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4rbp\" (UniqueName: \"kubernetes.io/projected/87b000f4-de74-4d7d-a412-bd87dbc26e0f-kube-api-access-l4rbp\") pod \"kuadrant-operator-catalog-8wcdx\" (UID: \"87b000f4-de74-4d7d-a412-bd87dbc26e0f\") " pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" Jun 13 01:05:44.570748 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.570723 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8wcdx"] Jun 13 01:05:44.668161 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.668115 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4rbp\" (UniqueName: \"kubernetes.io/projected/87b000f4-de74-4d7d-a412-bd87dbc26e0f-kube-api-access-l4rbp\") pod \"kuadrant-operator-catalog-8wcdx\" (UID: \"87b000f4-de74-4d7d-a412-bd87dbc26e0f\") " pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" Jun 13 01:05:44.693928 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.693899 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4rbp\" (UniqueName: \"kubernetes.io/projected/87b000f4-de74-4d7d-a412-bd87dbc26e0f-kube-api-access-l4rbp\") pod \"kuadrant-operator-catalog-8wcdx\" (UID: \"87b000f4-de74-4d7d-a412-bd87dbc26e0f\") " pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" Jun 13 01:05:44.868405 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.868314 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" Jun 13 01:05:44.922048 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.922021 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8wcdx"] Jun 13 01:05:44.996022 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:44.995932 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8wcdx"] Jun 13 01:05:44.998691 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:05:44.998662 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b000f4_de74_4d7d_a412_bd87dbc26e0f.slice/crio-ad7bbad206bfae3190a1a09aaed023187b7aac0e89557ff39e6096ef6734843a WatchSource:0}: Error finding container ad7bbad206bfae3190a1a09aaed023187b7aac0e89557ff39e6096ef6734843a: Status 404 returned error can't find the container with id ad7bbad206bfae3190a1a09aaed023187b7aac0e89557ff39e6096ef6734843a Jun 13 01:05:45.052133 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:45.052098 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" event={"ID":"87b000f4-de74-4d7d-a412-bd87dbc26e0f","Type":"ContainerStarted","Data":"ad7bbad206bfae3190a1a09aaed023187b7aac0e89557ff39e6096ef6734843a"} Jun 13 01:05:45.131372 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:45.131281 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8mztn"] Jun 13 01:05:45.133990 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:45.133968 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8mztn" Jun 13 01:05:45.142789 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:45.142755 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8mztn"] Jun 13 01:05:45.172159 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:45.172127 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mzf7\" (UniqueName: \"kubernetes.io/projected/561f4c8b-a710-4fe7-87e0-3b1c7270c6bc-kube-api-access-9mzf7\") pod \"kuadrant-operator-catalog-8mztn\" (UID: \"561f4c8b-a710-4fe7-87e0-3b1c7270c6bc\") " pod="kuadrant-system/kuadrant-operator-catalog-8mztn" Jun 13 01:05:45.272750 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:45.272711 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mzf7\" (UniqueName: \"kubernetes.io/projected/561f4c8b-a710-4fe7-87e0-3b1c7270c6bc-kube-api-access-9mzf7\") pod \"kuadrant-operator-catalog-8mztn\" (UID: \"561f4c8b-a710-4fe7-87e0-3b1c7270c6bc\") " pod="kuadrant-system/kuadrant-operator-catalog-8mztn" Jun 13 01:05:45.281299 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:45.281273 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mzf7\" (UniqueName: \"kubernetes.io/projected/561f4c8b-a710-4fe7-87e0-3b1c7270c6bc-kube-api-access-9mzf7\") pod \"kuadrant-operator-catalog-8mztn\" (UID: \"561f4c8b-a710-4fe7-87e0-3b1c7270c6bc\") " pod="kuadrant-system/kuadrant-operator-catalog-8mztn" Jun 13 01:05:45.443721 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:45.443636 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8mztn" Jun 13 01:05:45.683702 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:45.683675 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8mztn"] Jun 13 01:05:45.685954 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:05:45.685915 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561f4c8b_a710_4fe7_87e0_3b1c7270c6bc.slice/crio-5c3085dfa0bb100e257d95a9ec375fad6fb417849c0d16d04b23e7d6bdd99881 WatchSource:0}: Error finding container 5c3085dfa0bb100e257d95a9ec375fad6fb417849c0d16d04b23e7d6bdd99881: Status 404 returned error can't find the container with id 5c3085dfa0bb100e257d95a9ec375fad6fb417849c0d16d04b23e7d6bdd99881 Jun 13 01:05:46.059932 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:46.059891 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8mztn" event={"ID":"561f4c8b-a710-4fe7-87e0-3b1c7270c6bc","Type":"ContainerStarted","Data":"5c3085dfa0bb100e257d95a9ec375fad6fb417849c0d16d04b23e7d6bdd99881"} Jun 13 01:05:48.072270 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:48.072231 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" event={"ID":"87b000f4-de74-4d7d-a412-bd87dbc26e0f","Type":"ContainerStarted","Data":"6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14"} Jun 13 01:05:48.072728 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:48.072242 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" podUID="87b000f4-de74-4d7d-a412-bd87dbc26e0f" containerName="registry-server" containerID="cri-o://6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14" gracePeriod=2 Jun 13 01:05:48.073878 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:48.073848 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8mztn" event={"ID":"561f4c8b-a710-4fe7-87e0-3b1c7270c6bc","Type":"ContainerStarted","Data":"861b03d69bbfdfdc71b169cd6fcb194828769136e18ff9840a5987cc9be9f4eb"} Jun 13 01:05:48.090665 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:48.090606 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" podStartSLOduration=2.051537845 podStartE2EDuration="4.090587141s" totalImagesPullingTime="2.039049296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:05:48.088511052 +0000 UTC m=+587.662439963" watchObservedRunningTime="2026-06-13 01:05:48.090587141 +0000 UTC m=+587.664516029" Jun 13 01:05:48.106673 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:48.106630 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-8mztn" podStartSLOduration=1.751822013 podStartE2EDuration="3.106615095s" totalImagesPullingTime="1.354793082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:05:48.10534619 +0000 UTC m=+587.679275101" watchObservedRunningTime="2026-06-13 01:05:48.106615095 +0000 UTC m=+587.680543982" Jun 13 01:05:48.325535 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:48.325467 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" Jun 13 01:05:48.394031 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:48.393996 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4rbp\" (UniqueName: \"kubernetes.io/projected/87b000f4-de74-4d7d-a412-bd87dbc26e0f-kube-api-access-l4rbp\") pod \"87b000f4-de74-4d7d-a412-bd87dbc26e0f\" (UID: \"87b000f4-de74-4d7d-a412-bd87dbc26e0f\") " Jun 13 01:05:48.396184 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:48.396152 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87b000f4-de74-4d7d-a412-bd87dbc26e0f-kube-api-access-l4rbp" (OuterVolumeSpecName: "kube-api-access-l4rbp") pod "87b000f4-de74-4d7d-a412-bd87dbc26e0f" (UID: "87b000f4-de74-4d7d-a412-bd87dbc26e0f"). InnerVolumeSpecName "kube-api-access-l4rbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:05:48.495171 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:48.495143 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4rbp\" (UniqueName: \"kubernetes.io/projected/87b000f4-de74-4d7d-a412-bd87dbc26e0f-kube-api-access-l4rbp\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:05:49.079842 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:49.079812 2552 generic.go:356] "Generic (PLEG): container finished" podID="87b000f4-de74-4d7d-a412-bd87dbc26e0f" containerID="6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14" exitCode=0 Jun 13 01:05:49.080246 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:49.079848 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" Jun 13 01:05:49.080246 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:49.079893 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" event={"ID":"87b000f4-de74-4d7d-a412-bd87dbc26e0f","Type":"ContainerDied","Data":"6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14"} Jun 13 01:05:49.080246 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:49.079944 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8wcdx" event={"ID":"87b000f4-de74-4d7d-a412-bd87dbc26e0f","Type":"ContainerDied","Data":"ad7bbad206bfae3190a1a09aaed023187b7aac0e89557ff39e6096ef6734843a"} Jun 13 01:05:49.080246 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:49.079969 2552 scope.go:117] "RemoveContainer" containerID="6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14" Jun 13 01:05:49.088689 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:49.088674 2552 scope.go:117] "RemoveContainer" containerID="6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14" Jun 13 01:05:49.088962 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:05:49.088942 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14\": container with ID starting with 6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14 not found: ID does not exist" containerID="6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14" Jun 13 01:05:49.089037 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:49.088969 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14"} err="failed to get container status \"6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14\": rpc error: code = NotFound desc = could not find container \"6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14\": container with ID starting with 6ba8655a0ffbf158a908d7748c0cb8ab87bd60acb3f18ddfe432428560e75d14 not found: ID does not exist" Jun 13 01:05:49.100333 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:49.100309 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8wcdx"] Jun 13 01:05:49.105181 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:49.105162 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8wcdx"] Jun 13 01:05:50.854503 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:50.854472 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87b000f4-de74-4d7d-a412-bd87dbc26e0f" path="/var/lib/kubelet/pods/87b000f4-de74-4d7d-a412-bd87dbc26e0f/volumes" Jun 13 01:05:55.444588 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:55.444559 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-8mztn" Jun 13 01:05:55.445007 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:55.444612 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-8mztn" Jun 13 01:05:55.465672 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:55.465647 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-8mztn" Jun 13 01:05:56.138126 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:05:56.138098 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-8mztn" Jun 13 01:06:00.843318 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:00.843294 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:06:00.843678 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:00.843359 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:06:11.031754 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.031719 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8"] Jun 13 01:06:11.032198 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.032090 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87b000f4-de74-4d7d-a412-bd87dbc26e0f" containerName="registry-server" Jun 13 01:06:11.032198 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.032103 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="87b000f4-de74-4d7d-a412-bd87dbc26e0f" containerName="registry-server" Jun 13 01:06:11.032285 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.032202 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="87b000f4-de74-4d7d-a412-bd87dbc26e0f" containerName="registry-server" Jun 13 01:06:11.035071 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.035052 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" Jun 13 01:06:11.037565 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.037542 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 13 01:06:11.037711 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.037582 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-29c7r\"" Jun 13 01:06:11.045229 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.045207 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8"] Jun 13 01:06:11.060860 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.060834 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpmcp\" (UniqueName: \"kubernetes.io/projected/c8047eb1-2cd5-41b6-acc3-c7c753a2b615-kube-api-access-cpmcp\") pod \"dns-operator-controller-manager-864dd78c4c-8wmd8\" (UID: \"c8047eb1-2cd5-41b6-acc3-c7c753a2b615\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" Jun 13 01:06:11.161835 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.161801 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpmcp\" (UniqueName: \"kubernetes.io/projected/c8047eb1-2cd5-41b6-acc3-c7c753a2b615-kube-api-access-cpmcp\") pod \"dns-operator-controller-manager-864dd78c4c-8wmd8\" (UID: \"c8047eb1-2cd5-41b6-acc3-c7c753a2b615\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" Jun 13 01:06:11.177825 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.177771 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpmcp\" (UniqueName: \"kubernetes.io/projected/c8047eb1-2cd5-41b6-acc3-c7c753a2b615-kube-api-access-cpmcp\") pod \"dns-operator-controller-manager-864dd78c4c-8wmd8\" (UID: \"c8047eb1-2cd5-41b6-acc3-c7c753a2b615\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" Jun 13 01:06:11.346458 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.346367 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" Jun 13 01:06:11.490083 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:11.490049 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8"] Jun 13 01:06:11.493514 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:06:11.493487 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8047eb1_2cd5_41b6_acc3_c7c753a2b615.slice/crio-9ab75d4f028cc672e3ddab0563056612f808a3c81a12ab6d05c535c4a6c8941f WatchSource:0}: Error finding container 9ab75d4f028cc672e3ddab0563056612f808a3c81a12ab6d05c535c4a6c8941f: Status 404 returned error can't find the container with id 9ab75d4f028cc672e3ddab0563056612f808a3c81a12ab6d05c535c4a6c8941f Jun 13 01:06:12.189268 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:12.189104 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" event={"ID":"c8047eb1-2cd5-41b6-acc3-c7c753a2b615","Type":"ContainerStarted","Data":"9ab75d4f028cc672e3ddab0563056612f808a3c81a12ab6d05c535c4a6c8941f"} Jun 13 01:06:14.200384 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:14.200348 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" event={"ID":"c8047eb1-2cd5-41b6-acc3-c7c753a2b615","Type":"ContainerStarted","Data":"c56599f008fe9c6399435785ef8c97f36ac2451051278f2fba799e2e2fc1a9b2"} Jun 13 01:06:14.200890 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:14.200476 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" Jun 13 01:06:14.238793 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:14.238732 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" podStartSLOduration=1.465590935 podStartE2EDuration="3.238712376s" totalImagesPullingTime="1.773121441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:06:14.236510056 +0000 UTC m=+613.810438944" watchObservedRunningTime="2026-06-13 01:06:14.238712376 +0000 UTC m=+613.812641264" Jun 13 01:06:16.112474 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.112440 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp"] Jun 13 01:06:16.115950 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.115927 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:16.118331 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.118309 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-frk59\"" Jun 13 01:06:16.126791 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.126747 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp"] Jun 13 01:06:16.204709 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.204677 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" (UID: \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:16.204867 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.204814 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv498\" (UniqueName: \"kubernetes.io/projected/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-kube-api-access-lv498\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" (UID: \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:16.306177 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.306152 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv498\" (UniqueName: \"kubernetes.io/projected/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-kube-api-access-lv498\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" (UID: \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:16.306290 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.306202 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" (UID: \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:16.306545 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.306529 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" (UID: \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:16.314433 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.314409 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv498\" (UniqueName: \"kubernetes.io/projected/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-kube-api-access-lv498\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" (UID: \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:16.428388 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.428307 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:16.566260 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:16.566235 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp"] Jun 13 01:06:16.568403 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:06:16.568370 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d7f458_ed24_4eaf_94e4_1a2a46806d99.slice/crio-b5f564de1a3222d9c8939b8642c6f305500360230ad4c6191c7b52b85c7ada04 WatchSource:0}: Error finding container b5f564de1a3222d9c8939b8642c6f305500360230ad4c6191c7b52b85c7ada04: Status 404 returned error can't find the container with id b5f564de1a3222d9c8939b8642c6f305500360230ad4c6191c7b52b85c7ada04 Jun 13 01:06:17.215184 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:17.215148 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" event={"ID":"b1d7f458-ed24-4eaf-94e4-1a2a46806d99","Type":"ContainerStarted","Data":"b5f564de1a3222d9c8939b8642c6f305500360230ad4c6191c7b52b85c7ada04"} Jun 13 01:06:20.033041 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.033015 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fngbx"] Jun 13 01:06:20.037402 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.037385 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" Jun 13 01:06:20.041368 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.041345 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gcv8h\"" Jun 13 01:06:20.054215 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.054191 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fngbx"] Jun 13 01:06:20.137263 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.137234 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpgk8\" (UniqueName: \"kubernetes.io/projected/59571565-bb5a-475e-b92c-45b493d1dc54-kube-api-access-jpgk8\") pod \"authorino-operator-75cc8d5cb7-fngbx\" (UID: \"59571565-bb5a-475e-b92c-45b493d1dc54\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" Jun 13 01:06:20.231124 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.231094 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" event={"ID":"b1d7f458-ed24-4eaf-94e4-1a2a46806d99","Type":"ContainerStarted","Data":"3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419"} Jun 13 01:06:20.231248 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.231153 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:20.238587 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.238524 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpgk8\" (UniqueName: \"kubernetes.io/projected/59571565-bb5a-475e-b92c-45b493d1dc54-kube-api-access-jpgk8\") pod \"authorino-operator-75cc8d5cb7-fngbx\" (UID: \"59571565-bb5a-475e-b92c-45b493d1dc54\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" Jun 13 01:06:20.250009 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.249984 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpgk8\" (UniqueName: \"kubernetes.io/projected/59571565-bb5a-475e-b92c-45b493d1dc54-kube-api-access-jpgk8\") pod \"authorino-operator-75cc8d5cb7-fngbx\" (UID: \"59571565-bb5a-475e-b92c-45b493d1dc54\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" Jun 13 01:06:20.253214 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.253177 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" podStartSLOduration=0.836777447 podStartE2EDuration="4.253162489s" totalImagesPullingTime="3.416385042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:06:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:06:20.252182368 +0000 UTC m=+619.826111270" watchObservedRunningTime="2026-06-13 01:06:20.253162489 +0000 UTC m=+619.827091423" Jun 13 01:06:20.347413 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.347385 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" Jun 13 01:06:20.487141 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:20.487086 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fngbx"] Jun 13 01:06:20.489417 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:06:20.489344 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59571565_bb5a_475e_b92c_45b493d1dc54.slice/crio-189a1202d531fe3122eaa4d293fd8e47c00dad0c896d6ede18e3815b4d69ccf7 WatchSource:0}: Error finding container 189a1202d531fe3122eaa4d293fd8e47c00dad0c896d6ede18e3815b4d69ccf7: Status 404 returned error can't find the container with id 189a1202d531fe3122eaa4d293fd8e47c00dad0c896d6ede18e3815b4d69ccf7 Jun 13 01:06:21.239276 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:21.239196 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" event={"ID":"59571565-bb5a-475e-b92c-45b493d1dc54","Type":"ContainerStarted","Data":"189a1202d531fe3122eaa4d293fd8e47c00dad0c896d6ede18e3815b4d69ccf7"} Jun 13 01:06:23.251633 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:23.251599 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" event={"ID":"59571565-bb5a-475e-b92c-45b493d1dc54","Type":"ContainerStarted","Data":"8ca74f5b61823ae885694222eb8b3ebbc5f5b2dce9f178b49ab1eb5faeed5ea3"} Jun 13 01:06:23.252130 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:23.251719 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" Jun 13 01:06:23.295608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:23.295559 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" podStartSLOduration=1.262478878 podStartE2EDuration="3.295547095s" totalImagesPullingTime="2.033068217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:06:23.293965978 +0000 UTC m=+622.867895000" watchObservedRunningTime="2026-06-13 01:06:23.295547095 +0000 UTC m=+622.869475983" Jun 13 01:06:25.207048 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:25.207009 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8wmd8" Jun 13 01:06:31.241993 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:31.241965 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:32.867042 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:32.867012 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl"] Jun 13 01:06:32.870442 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:32.870423 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:32.890228 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:32.890206 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl"] Jun 13 01:06:32.948315 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:32.948293 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjqxg\" (UniqueName: \"kubernetes.io/projected/e38c2c59-4126-4f0e-8406-8d01d71e3071-kube-api-access-xjqxg\") pod \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" (UID: \"e38c2c59-4126-4f0e-8406-8d01d71e3071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:32.948425 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:32.948347 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e38c2c59-4126-4f0e-8406-8d01d71e3071-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" (UID: \"e38c2c59-4126-4f0e-8406-8d01d71e3071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:33.014263 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.014241 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl"] Jun 13 01:06:33.014499 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.014477 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-xjqxg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" podUID="e38c2c59-4126-4f0e-8406-8d01d71e3071" Jun 13 01:06:33.024399 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.024363 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl"] Jun 13 01:06:33.038318 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.038293 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp"] Jun 13 01:06:33.038485 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.038465 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" containerName="manager" containerID="cri-o://3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419" gracePeriod=2 Jun 13 01:06:33.042684 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.042657 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb"] Jun 13 01:06:33.046389 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.046371 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:33.048826 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.048800 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e38c2c59-4126-4f0e-8406-8d01d71e3071-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" (UID: \"e38c2c59-4126-4f0e-8406-8d01d71e3071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:33.048928 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.048881 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjqxg\" (UniqueName: \"kubernetes.io/projected/e38c2c59-4126-4f0e-8406-8d01d71e3071-kube-api-access-xjqxg\") pod \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" (UID: \"e38c2c59-4126-4f0e-8406-8d01d71e3071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:33.049102 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.049082 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e38c2c59-4126-4f0e-8406-8d01d71e3071-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" (UID: \"e38c2c59-4126-4f0e-8406-8d01d71e3071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:33.053504 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.053480 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xjqxg for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-dgfdl" not found Jun 13 01:06:33.053611 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.053556 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e38c2c59-4126-4f0e-8406-8d01d71e3071-kube-api-access-xjqxg podName:e38c2c59-4126-4f0e-8406-8d01d71e3071 nodeName:}" failed. No retries permitted until 2026-06-13 01:06:33.553537999 +0000 UTC m=+633.127466880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xjqxg" (UniqueName: "kubernetes.io/projected/e38c2c59-4126-4f0e-8406-8d01d71e3071-kube-api-access-xjqxg") pod "kuadrant-operator-controller-manager-55467d9f7-dgfdl" (UID: "e38c2c59-4126-4f0e-8406-8d01d71e3071") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-dgfdl" not found Jun 13 01:06:33.054895 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.054871 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp"] Jun 13 01:06:33.066282 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.066255 2552 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\": pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:33.068755 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.068706 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:33.088340 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.088314 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb"] Jun 13 01:06:33.091182 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.091161 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn"] Jun 13 01:06:33.091501 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.091488 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" containerName="manager" Jun 13 01:06:33.091501 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.091501 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" containerName="manager" Jun 13 01:06:33.091631 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.091557 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" containerName="manager" Jun 13 01:06:33.094475 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.094456 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:33.141843 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.141818 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn"] Jun 13 01:06:33.149354 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.149331 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zbszb\" (UID: \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:33.149461 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.149357 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9gfx\" (UniqueName: \"kubernetes.io/projected/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-kube-api-access-h9gfx\") pod \"kuadrant-operator-controller-manager-55467d9f7-zbszb\" (UID: \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:33.149461 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.149385 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82b2a102-aa6c-4656-afe7-f7580fa7b67b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9jgn\" (UID: \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:33.149588 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.149534 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cp55\" (UniqueName: \"kubernetes.io/projected/82b2a102-aa6c-4656-afe7-f7580fa7b67b-kube-api-access-2cp55\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9jgn\" (UID: \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:33.185849 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.185753 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:33.252548 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.240695 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn"] Jun 13 01:06:33.252548 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.252453 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-2cp55], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" podUID="82b2a102-aa6c-4656-afe7-f7580fa7b67b" Jun 13 01:06:33.252838 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.252812 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cp55\" (UniqueName: \"kubernetes.io/projected/82b2a102-aa6c-4656-afe7-f7580fa7b67b-kube-api-access-2cp55\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9jgn\" (UID: \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:33.252970 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.252915 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zbszb\" (UID: \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:33.253082 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.252945 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9gfx\" (UniqueName: \"kubernetes.io/projected/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-kube-api-access-h9gfx\") pod \"kuadrant-operator-controller-manager-55467d9f7-zbszb\" (UID: \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:33.253187 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.253129 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82b2a102-aa6c-4656-afe7-f7580fa7b67b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9jgn\" (UID: \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:33.253328 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.253310 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zbszb\" (UID: \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:33.253608 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.253585 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82b2a102-aa6c-4656-afe7-f7580fa7b67b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9jgn\" (UID: \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:33.265945 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.265926 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cp55\" (UniqueName: \"kubernetes.io/projected/82b2a102-aa6c-4656-afe7-f7580fa7b67b-kube-api-access-2cp55\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9jgn\" (UID: \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:33.272399 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.272378 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9gfx\" (UniqueName: \"kubernetes.io/projected/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-kube-api-access-h9gfx\") pod \"kuadrant-operator-controller-manager-55467d9f7-zbszb\" (UID: \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:33.296568 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.296551 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:33.299218 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.299194 2552 generic.go:356] "Generic (PLEG): container finished" podID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" containerID="3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419" exitCode=0 Jun 13 01:06:33.299305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.299235 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:33.299305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.299289 2552 scope.go:117] "RemoveContainer" containerID="3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419" Jun 13 01:06:33.299390 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.299331 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:33.299390 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.299343 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:33.300260 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.300231 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:33.302225 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.302196 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:33.304256 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.304229 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="e38c2c59-4126-4f0e-8406-8d01d71e3071" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:33.305582 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.305567 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:33.307715 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.307700 2552 scope.go:117] "RemoveContainer" containerID="3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419" Jun 13 01:06:33.307974 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.307957 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419\": container with ID starting with 3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419 not found: ID does not exist" containerID="3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419" Jun 13 01:06:33.308031 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.307979 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419"} err="failed to get container status \"3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419\": rpc error: code = NotFound desc = could not find container \"3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419\": container with ID starting with 3cb11762bccd4f381cfba3abeebf7124cce1d18a0e640112aa7a4df73ff6f419 not found: ID does not exist" Jun 13 01:06:33.310404 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.310390 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:33.314304 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.314283 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9"] Jun 13 01:06:33.317698 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.317680 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:33.331204 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.331179 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:33.333604 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.333584 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="e38c2c59-4126-4f0e-8406-8d01d71e3071" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:33.335423 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.335403 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9"] Jun 13 01:06:33.423277 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.423211 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:33.455174 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455150 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-extensions-socket-volume\") pod \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\" (UID: \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\") " Jun 13 01:06:33.455289 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455189 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e38c2c59-4126-4f0e-8406-8d01d71e3071-extensions-socket-volume\") pod \"e38c2c59-4126-4f0e-8406-8d01d71e3071\" (UID: \"e38c2c59-4126-4f0e-8406-8d01d71e3071\") " Jun 13 01:06:33.455289 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455230 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82b2a102-aa6c-4656-afe7-f7580fa7b67b-extensions-socket-volume\") pod \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\" (UID: \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\") " Jun 13 01:06:33.455390 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455312 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cp55\" (UniqueName: \"kubernetes.io/projected/82b2a102-aa6c-4656-afe7-f7580fa7b67b-kube-api-access-2cp55\") pod \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\" (UID: \"82b2a102-aa6c-4656-afe7-f7580fa7b67b\") " Jun 13 01:06:33.455390 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455355 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lv498\" (UniqueName: \"kubernetes.io/projected/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-kube-api-access-lv498\") pod \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\" (UID: \"b1d7f458-ed24-4eaf-94e4-1a2a46806d99\") " Jun 13 01:06:33.455540 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455459 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e38c2c59-4126-4f0e-8406-8d01d71e3071-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e38c2c59-4126-4f0e-8406-8d01d71e3071" (UID: "e38c2c59-4126-4f0e-8406-8d01d71e3071"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:06:33.455540 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455483 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dvkx9\" (UID: \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:33.455644 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455595 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b2a102-aa6c-4656-afe7-f7580fa7b67b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "82b2a102-aa6c-4656-afe7-f7580fa7b67b" (UID: "82b2a102-aa6c-4656-afe7-f7580fa7b67b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:06:33.455644 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455621 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-698dq\" (UniqueName: \"kubernetes.io/projected/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-kube-api-access-698dq\") pod \"kuadrant-operator-controller-manager-86878c769f-dvkx9\" (UID: \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:33.455736 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455676 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjqxg\" (UniqueName: \"kubernetes.io/projected/e38c2c59-4126-4f0e-8406-8d01d71e3071-kube-api-access-xjqxg\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:06:33.455736 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455686 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b1d7f458-ed24-4eaf-94e4-1a2a46806d99" (UID: "b1d7f458-ed24-4eaf-94e4-1a2a46806d99"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:06:33.455736 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455694 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e38c2c59-4126-4f0e-8406-8d01d71e3071-extensions-socket-volume\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:06:33.455736 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.455727 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/82b2a102-aa6c-4656-afe7-f7580fa7b67b-extensions-socket-volume\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:06:33.457354 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.457316 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b2a102-aa6c-4656-afe7-f7580fa7b67b-kube-api-access-2cp55" (OuterVolumeSpecName: "kube-api-access-2cp55") pod "82b2a102-aa6c-4656-afe7-f7580fa7b67b" (UID: "82b2a102-aa6c-4656-afe7-f7580fa7b67b"). InnerVolumeSpecName "kube-api-access-2cp55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:06:33.457513 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.457498 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-kube-api-access-lv498" (OuterVolumeSpecName: "kube-api-access-lv498") pod "b1d7f458-ed24-4eaf-94e4-1a2a46806d99" (UID: "b1d7f458-ed24-4eaf-94e4-1a2a46806d99"). InnerVolumeSpecName "kube-api-access-lv498". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:06:33.555864 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.555752 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb"] Jun 13 01:06:33.556152 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.556135 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-698dq\" (UniqueName: \"kubernetes.io/projected/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-kube-api-access-698dq\") pod \"kuadrant-operator-controller-manager-86878c769f-dvkx9\" (UID: \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:33.556380 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.556185 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dvkx9\" (UID: \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:33.556380 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.556257 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-extensions-socket-volume\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:06:33.556380 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.556274 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cp55\" (UniqueName: \"kubernetes.io/projected/82b2a102-aa6c-4656-afe7-f7580fa7b67b-kube-api-access-2cp55\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:06:33.556380 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.556288 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv498\" (UniqueName: \"kubernetes.io/projected/b1d7f458-ed24-4eaf-94e4-1a2a46806d99-kube-api-access-lv498\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:06:33.556662 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.556525 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dvkx9\" (UID: \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:33.558769 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:06:33.558747 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9802e7_8e07_4bd9_8eae_73e41d8d7ad1.slice/crio-a5f7bf631895a63df7025d9e983f0a271087e64ca60763d7b9b602376aa304b3 WatchSource:0}: Error finding container a5f7bf631895a63df7025d9e983f0a271087e64ca60763d7b9b602376aa304b3: Status 404 returned error can't find the container with id a5f7bf631895a63df7025d9e983f0a271087e64ca60763d7b9b602376aa304b3 Jun 13 01:06:33.566635 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.566618 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-698dq\" (UniqueName: \"kubernetes.io/projected/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-kube-api-access-698dq\") pod \"kuadrant-operator-controller-manager-86878c769f-dvkx9\" (UID: \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:33.611976 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.611949 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:33.613924 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:33.613902 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="e38c2c59-4126-4f0e-8406-8d01d71e3071" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:33.626059 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.626041 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:33.757822 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:33.757793 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9"] Jun 13 01:06:33.761093 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:06:33.761070 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b991d83_6dd3_4b34_bdda_e379d7dce2eb.slice/crio-3c00e117da7761dd424d4e1fdfecc994e1438fc455a8e67ac34e036f1decb8d9 WatchSource:0}: Error finding container 3c00e117da7761dd424d4e1fdfecc994e1438fc455a8e67ac34e036f1decb8d9: Status 404 returned error can't find the container with id 3c00e117da7761dd424d4e1fdfecc994e1438fc455a8e67ac34e036f1decb8d9 Jun 13 01:06:34.258540 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.258507 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fngbx" Jun 13 01:06:34.260899 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:34.260875 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="e38c2c59-4126-4f0e-8406-8d01d71e3071" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:34.283451 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:34.283415 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:34.306924 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.306894 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" event={"ID":"3b991d83-6dd3-4b34-bdda-e379d7dce2eb","Type":"ContainerStarted","Data":"aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378"} Jun 13 01:06:34.307094 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.306944 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" event={"ID":"3b991d83-6dd3-4b34-bdda-e379d7dce2eb","Type":"ContainerStarted","Data":"3c00e117da7761dd424d4e1fdfecc994e1438fc455a8e67ac34e036f1decb8d9"} Jun 13 01:06:34.307094 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.307069 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:34.308701 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.308681 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn" Jun 13 01:06:34.308861 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.308681 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" event={"ID":"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1","Type":"ContainerStarted","Data":"aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643"} Jun 13 01:06:34.308861 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.308800 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:34.308861 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.308808 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:34.308861 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.308834 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" event={"ID":"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1","Type":"ContainerStarted","Data":"a5f7bf631895a63df7025d9e983f0a271087e64ca60763d7b9b602376aa304b3"} Jun 13 01:06:34.309082 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:34.309015 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:34.311108 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:34.311038 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="e38c2c59-4126-4f0e-8406-8d01d71e3071" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:34.338531 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.338490 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" podStartSLOduration=1.338477876 podStartE2EDuration="1.338477876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:06:34.336283134 +0000 UTC m=+633.910212046" watchObservedRunningTime="2026-06-13 01:06:34.338477876 +0000 UTC m=+633.912406764" Jun 13 01:06:34.355933 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.355894 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" podStartSLOduration=1.355883403 podStartE2EDuration="1.355883403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:06:34.355296577 +0000 UTC m=+633.929225464" watchObservedRunningTime="2026-06-13 01:06:34.355883403 +0000 UTC m=+633.929812288" Jun 13 01:06:34.357451 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:34.357414 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qgqpp\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qgqpp" Jun 13 01:06:34.359148 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:34.359125 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-dgfdl\" is forbidden: User \"system:node:ip-10-0-137-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-107.ec2.internal' and this object" podUID="e38c2c59-4126-4f0e-8406-8d01d71e3071" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-dgfdl" Jun 13 01:06:34.398116 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.398090 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn"] Jun 13 01:06:34.411707 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.411679 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9jgn"] Jun 13 01:06:34.856183 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.856151 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82b2a102-aa6c-4656-afe7-f7580fa7b67b" path="/var/lib/kubelet/pods/82b2a102-aa6c-4656-afe7-f7580fa7b67b/volumes" Jun 13 01:06:34.856412 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.856399 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1d7f458-ed24-4eaf-94e4-1a2a46806d99" path="/var/lib/kubelet/pods/b1d7f458-ed24-4eaf-94e4-1a2a46806d99/volumes" Jun 13 01:06:34.856691 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:34.856679 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e38c2c59-4126-4f0e-8406-8d01d71e3071" path="/var/lib/kubelet/pods/e38c2c59-4126-4f0e-8406-8d01d71e3071/volumes" Jun 13 01:06:45.314918 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.314888 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:45.315304 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.314960 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:06:45.416853 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.416821 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb"] Jun 13 01:06:45.417060 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.417040 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" podUID="bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1" containerName="manager" containerID="cri-o://aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643" gracePeriod=10 Jun 13 01:06:45.670754 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.670729 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:45.853449 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.853370 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-extensions-socket-volume\") pod \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\" (UID: \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\") " Jun 13 01:06:45.853449 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.853437 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9gfx\" (UniqueName: \"kubernetes.io/projected/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-kube-api-access-h9gfx\") pod \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\" (UID: \"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1\") " Jun 13 01:06:45.853801 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.853753 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1" (UID: "bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:06:45.855293 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.855273 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-kube-api-access-h9gfx" (OuterVolumeSpecName: "kube-api-access-h9gfx") pod "bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1" (UID: "bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1"). InnerVolumeSpecName "kube-api-access-h9gfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:06:45.954393 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.954371 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-extensions-socket-volume\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:06:45.954393 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:45.954390 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9gfx\" (UniqueName: \"kubernetes.io/projected/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1-kube-api-access-h9gfx\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:06:46.375184 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.375152 2552 generic.go:356] "Generic (PLEG): container finished" podID="bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1" containerID="aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643" exitCode=0 Jun 13 01:06:46.375591 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.375193 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" Jun 13 01:06:46.375591 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.375246 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" event={"ID":"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1","Type":"ContainerDied","Data":"aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643"} Jun 13 01:06:46.375591 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.375292 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb" event={"ID":"bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1","Type":"ContainerDied","Data":"a5f7bf631895a63df7025d9e983f0a271087e64ca60763d7b9b602376aa304b3"} Jun 13 01:06:46.375591 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.375319 2552 scope.go:117] "RemoveContainer" containerID="aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643" Jun 13 01:06:46.383661 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.383644 2552 scope.go:117] "RemoveContainer" containerID="aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643" Jun 13 01:06:46.383948 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:06:46.383928 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643\": container with ID starting with aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643 not found: ID does not exist" containerID="aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643" Jun 13 01:06:46.384007 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.383955 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643"} err="failed to get container status \"aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643\": rpc error: code = NotFound desc = could not find container \"aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643\": container with ID starting with aa6424cd6920349feae54a9a41a93f9664a583dcc35eff150d65436c8860b643 not found: ID does not exist" Jun 13 01:06:46.399356 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.399334 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb"] Jun 13 01:06:46.403521 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.403503 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zbszb"] Jun 13 01:06:46.855585 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:06:46.855557 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1" path="/var/lib/kubelet/pods/bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1/volumes" Jun 13 01:07:01.759854 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.759818 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7"] Jun 13 01:07:01.760367 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.760351 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1" containerName="manager" Jun 13 01:07:01.760431 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.760371 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1" containerName="manager" Jun 13 01:07:01.760496 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.760485 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf9802e7-8e07-4bd9-8eae-73e41d8d7ad1" containerName="manager" Jun 13 01:07:01.765907 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.765886 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.768412 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.768392 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4n4gh\"" Jun 13 01:07:01.778022 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.777997 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7"] Jun 13 01:07:01.865948 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.865917 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.866125 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.866062 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.866410 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.866384 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.866551 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.866422 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtwp6\" (UniqueName: \"kubernetes.io/projected/f4876ece-d3c1-45a5-9c59-300d782c7bdc-kube-api-access-xtwp6\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.866551 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.866458 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.866725 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.866708 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.866817 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.866737 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.866817 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.866760 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.866941 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.866838 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.967466 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.967437 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.967610 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.967485 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.967610 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.967510 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.967610 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.967525 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtwp6\" (UniqueName: \"kubernetes.io/projected/f4876ece-d3c1-45a5-9c59-300d782c7bdc-kube-api-access-xtwp6\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.967610 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.967543 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.967610 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.967580 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.967610 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.967594 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.967976 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.967617 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.967976 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.967650 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.968085 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.968008 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.968303 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.968280 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.968720 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.968653 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.968720 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.968398 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.968720 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.968634 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.969951 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.969931 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.970227 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.970211 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.978703 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.978682 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4876ece-d3c1-45a5-9c59-300d782c7bdc-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:01.979543 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:01.979515 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtwp6\" (UniqueName: \"kubernetes.io/projected/f4876ece-d3c1-45a5-9c59-300d782c7bdc-kube-api-access-xtwp6\") pod \"maas-default-gateway-openshift-default-8559cd5744-pdnf7\" (UID: \"f4876ece-d3c1-45a5-9c59-300d782c7bdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:02.075889 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:02.075854 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:02.222176 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:02.222152 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7"] Jun 13 01:07:02.224074 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:07:02.224036 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4876ece_d3c1_45a5_9c59_300d782c7bdc.slice/crio-87309da6c0e747655c199f5f16a4673ca184bcea9bfcafe78fa7c461683be8d5 WatchSource:0}: Error finding container 87309da6c0e747655c199f5f16a4673ca184bcea9bfcafe78fa7c461683be8d5: Status 404 returned error can't find the container with id 87309da6c0e747655c199f5f16a4673ca184bcea9bfcafe78fa7c461683be8d5 Jun 13 01:07:02.226188 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:02.226160 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 13 01:07:02.226275 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:02.226220 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 13 01:07:02.226275 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:02.226252 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 13 01:07:02.452761 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:02.452677 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" event={"ID":"f4876ece-d3c1-45a5-9c59-300d782c7bdc","Type":"ContainerStarted","Data":"7a4cb3191b9a3c9084aeb15a749804d6b2c1b6ebb3e5e00928cb050e9d419b5c"} Jun 13 01:07:02.452761 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:02.452729 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" event={"ID":"f4876ece-d3c1-45a5-9c59-300d782c7bdc","Type":"ContainerStarted","Data":"87309da6c0e747655c199f5f16a4673ca184bcea9bfcafe78fa7c461683be8d5"} Jun 13 01:07:02.473712 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:02.473659 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" podStartSLOduration=1.473644283 podStartE2EDuration="1.473644283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:07:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:07:02.472423313 +0000 UTC m=+662.046352202" watchObservedRunningTime="2026-06-13 01:07:02.473644283 +0000 UTC m=+662.047573171" Jun 13 01:07:03.076899 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:03.076861 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:03.081667 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:03.081627 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:03.457621 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:03.457538 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:03.458604 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:03.458583 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:07:04.495585 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:04.495557 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-tlgsb"] Jun 13 01:07:04.498221 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:04.498201 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tlgsb" Jun 13 01:07:04.501830 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:04.501811 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b7jx6\"" Jun 13 01:07:04.508356 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:04.508336 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tlgsb"] Jun 13 01:07:04.591069 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:04.591041 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drfzv\" (UniqueName: \"kubernetes.io/projected/eea04aeb-7d42-49af-b066-ce0d4c6ee271-kube-api-access-drfzv\") pod \"authorino-75d556bd77-tlgsb\" (UID: \"eea04aeb-7d42-49af-b066-ce0d4c6ee271\") " pod="kuadrant-system/authorino-75d556bd77-tlgsb" Jun 13 01:07:04.692110 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:04.692084 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drfzv\" (UniqueName: \"kubernetes.io/projected/eea04aeb-7d42-49af-b066-ce0d4c6ee271-kube-api-access-drfzv\") pod \"authorino-75d556bd77-tlgsb\" (UID: \"eea04aeb-7d42-49af-b066-ce0d4c6ee271\") " pod="kuadrant-system/authorino-75d556bd77-tlgsb" Jun 13 01:07:04.700389 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:04.700361 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drfzv\" (UniqueName: \"kubernetes.io/projected/eea04aeb-7d42-49af-b066-ce0d4c6ee271-kube-api-access-drfzv\") pod \"authorino-75d556bd77-tlgsb\" (UID: \"eea04aeb-7d42-49af-b066-ce0d4c6ee271\") " pod="kuadrant-system/authorino-75d556bd77-tlgsb" Jun 13 01:07:04.808960 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:04.808934 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tlgsb" Jun 13 01:07:04.938079 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:04.938050 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tlgsb"] Jun 13 01:07:04.941256 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:07:04.941220 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeea04aeb_7d42_49af_b066_ce0d4c6ee271.slice/crio-4264ac4a04ed9901bc0292dd2e7ee7332174e510da4b14a28ba7fe26133ea74a WatchSource:0}: Error finding container 4264ac4a04ed9901bc0292dd2e7ee7332174e510da4b14a28ba7fe26133ea74a: Status 404 returned error can't find the container with id 4264ac4a04ed9901bc0292dd2e7ee7332174e510da4b14a28ba7fe26133ea74a Jun 13 01:07:05.474311 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:05.474252 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tlgsb" event={"ID":"eea04aeb-7d42-49af-b066-ce0d4c6ee271","Type":"ContainerStarted","Data":"4264ac4a04ed9901bc0292dd2e7ee7332174e510da4b14a28ba7fe26133ea74a"} Jun 13 01:07:08.491057 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:08.491018 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tlgsb" event={"ID":"eea04aeb-7d42-49af-b066-ce0d4c6ee271","Type":"ContainerStarted","Data":"bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6"} Jun 13 01:07:08.505933 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:07:08.505884 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-tlgsb" podStartSLOduration=1.41644058 podStartE2EDuration="4.505868705s" totalImagesPullingTime="3.089428125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:07:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:07:08.504963865 +0000 UTC m=+668.078892753" watchObservedRunningTime="2026-06-13 01:07:08.505868705 +0000 UTC m=+668.079797592" Jun 13 01:08:00.699992 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.699919 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 13 01:08:00.702265 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.702247 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 13 01:08:00.704460 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.704430 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 13 01:08:00.704556 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.704482 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 13 01:08:00.704556 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.704437 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 13 01:08:00.705470 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.705452 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-wzf59\"" Jun 13 01:08:00.711531 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.711509 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 13 01:08:00.842454 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.842430 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8gcw\" (UniqueName: \"kubernetes.io/projected/69bca32c-6609-4411-b8f5-03099d6f743b-kube-api-access-r8gcw\") pod \"maas-keycloak-0\" (UID: \"69bca32c-6609-4411-b8f5-03099d6f743b\") " pod="keycloak-system/maas-keycloak-0" Jun 13 01:08:00.942971 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.942944 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8gcw\" (UniqueName: \"kubernetes.io/projected/69bca32c-6609-4411-b8f5-03099d6f743b-kube-api-access-r8gcw\") pod \"maas-keycloak-0\" (UID: \"69bca32c-6609-4411-b8f5-03099d6f743b\") " pod="keycloak-system/maas-keycloak-0" Jun 13 01:08:00.951105 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.951041 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 13 01:08:00.961945 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.961925 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 13 01:08:00.972997 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:00.972974 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8gcw\" (UniqueName: \"kubernetes.io/projected/69bca32c-6609-4411-b8f5-03099d6f743b-kube-api-access-r8gcw\") pod \"maas-keycloak-0\" (UID: \"69bca32c-6609-4411-b8f5-03099d6f743b\") " pod="keycloak-system/maas-keycloak-0" Jun 13 01:08:01.013960 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:01.013941 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-wzf59\"" Jun 13 01:08:01.022165 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:01.022141 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 13 01:08:01.145914 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:01.145892 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 13 01:08:01.147604 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:08:01.147579 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69bca32c_6609_4411_b8f5_03099d6f743b.slice/crio-7dd842f23b0f8da410be26bf2c5138204d0c7d2c85cfab62a923247937bfd66d WatchSource:0}: Error finding container 7dd842f23b0f8da410be26bf2c5138204d0c7d2c85cfab62a923247937bfd66d: Status 404 returned error can't find the container with id 7dd842f23b0f8da410be26bf2c5138204d0c7d2c85cfab62a923247937bfd66d Jun 13 01:08:01.149059 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:01.149038 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 13 01:08:01.743982 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:01.743903 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"69bca32c-6609-4411-b8f5-03099d6f743b","Type":"ContainerStarted","Data":"7dd842f23b0f8da410be26bf2c5138204d0c7d2c85cfab62a923247937bfd66d"} Jun 13 01:08:05.421500 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:05.421481 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 13 01:08:05.768250 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:05.768146 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"69bca32c-6609-4411-b8f5-03099d6f743b","Type":"ContainerStarted","Data":"c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500"} Jun 13 01:08:05.802902 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:05.802849 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.533222327 podStartE2EDuration="5.802834818s" totalImagesPullingTime="4.269612491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:08:05.801757738 +0000 UTC m=+725.375686625" watchObservedRunningTime="2026-06-13 01:08:05.802834818 +0000 UTC m=+725.376763704" Jun 13 01:08:06.023336 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:06.023253 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 13 01:08:06.024919 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:06.024892 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:07.023244 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:07.023188 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:08.023101 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:08.023063 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:09.023483 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:09.023440 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:10.024267 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:10.024153 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:11.022290 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:11.022260 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 13 01:08:11.022702 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:11.022651 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:12.022684 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:12.022628 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:13.022645 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:13.022594 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:14.023183 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:14.023133 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:15.022976 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:15.022931 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:16.022634 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:16.022587 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:17.023267 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:17.023221 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.41:9000/health/started\": dial tcp 10.132.0.41:9000: connect: connection refused" Jun 13 01:08:18.115254 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:18.115219 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 13 01:08:18.131082 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:18.131036 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 13 01:08:28.121354 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:28.121321 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 13 01:08:28.987692 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:28.987639 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-87k2f"] Jun 13 01:08:28.996373 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:28.996351 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-87k2f"] Jun 13 01:08:28.996541 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:28.996452 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-87k2f" Jun 13 01:08:29.134035 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.134006 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcs6r\" (UniqueName: \"kubernetes.io/projected/901be467-bb07-4810-90b7-09169fee8905-kube-api-access-xcs6r\") pod \"authorino-5d4b67c446-87k2f\" (UID: \"901be467-bb07-4810-90b7-09169fee8905\") " pod="kuadrant-system/authorino-5d4b67c446-87k2f" Jun 13 01:08:29.212832 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.212802 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-87k2f"] Jun 13 01:08:29.213064 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:08:29.213045 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xcs6r], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-87k2f" podUID="901be467-bb07-4810-90b7-09169fee8905" Jun 13 01:08:29.232015 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.231991 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-784b644797-cwkv8"] Jun 13 01:08:29.234211 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.234196 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-784b644797-cwkv8" Jun 13 01:08:29.234426 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.234406 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcs6r\" (UniqueName: \"kubernetes.io/projected/901be467-bb07-4810-90b7-09169fee8905-kube-api-access-xcs6r\") pod \"authorino-5d4b67c446-87k2f\" (UID: \"901be467-bb07-4810-90b7-09169fee8905\") " pod="kuadrant-system/authorino-5d4b67c446-87k2f" Jun 13 01:08:29.243951 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.243929 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-784b644797-cwkv8"] Jun 13 01:08:29.246113 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.246087 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcs6r\" (UniqueName: \"kubernetes.io/projected/901be467-bb07-4810-90b7-09169fee8905-kube-api-access-xcs6r\") pod \"authorino-5d4b67c446-87k2f\" (UID: \"901be467-bb07-4810-90b7-09169fee8905\") " pod="kuadrant-system/authorino-5d4b67c446-87k2f" Jun 13 01:08:29.335207 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.335172 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvsfq\" (UniqueName: \"kubernetes.io/projected/55679c68-0cfc-4513-8bac-15dbb323dbbe-kube-api-access-qvsfq\") pod \"authorino-784b644797-cwkv8\" (UID: \"55679c68-0cfc-4513-8bac-15dbb323dbbe\") " pod="kuadrant-system/authorino-784b644797-cwkv8" Jun 13 01:08:29.369171 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.369139 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-784b644797-cwkv8"] Jun 13 01:08:29.369391 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:08:29.369372 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qvsfq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-784b644797-cwkv8" podUID="55679c68-0cfc-4513-8bac-15dbb323dbbe" Jun 13 01:08:29.396354 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.396324 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5ccf5b4977-md4km"] Jun 13 01:08:29.398875 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.398854 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:08:29.401154 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.401131 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 13 01:08:29.407599 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.407575 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ccf5b4977-md4km"] Jun 13 01:08:29.435879 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.435852 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvsfq\" (UniqueName: \"kubernetes.io/projected/55679c68-0cfc-4513-8bac-15dbb323dbbe-kube-api-access-qvsfq\") pod \"authorino-784b644797-cwkv8\" (UID: \"55679c68-0cfc-4513-8bac-15dbb323dbbe\") " pod="kuadrant-system/authorino-784b644797-cwkv8" Jun 13 01:08:29.444313 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.444284 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvsfq\" (UniqueName: \"kubernetes.io/projected/55679c68-0cfc-4513-8bac-15dbb323dbbe-kube-api-access-qvsfq\") pod \"authorino-784b644797-cwkv8\" (UID: \"55679c68-0cfc-4513-8bac-15dbb323dbbe\") " pod="kuadrant-system/authorino-784b644797-cwkv8" Jun 13 01:08:29.536846 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.536732 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0911159d-bfe3-41f2-a186-01866d7b98a7-tls-cert\") pod \"authorino-5ccf5b4977-md4km\" (UID: \"0911159d-bfe3-41f2-a186-01866d7b98a7\") " pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:08:29.536846 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.536770 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9ggf\" (UniqueName: \"kubernetes.io/projected/0911159d-bfe3-41f2-a186-01866d7b98a7-kube-api-access-p9ggf\") pod \"authorino-5ccf5b4977-md4km\" (UID: \"0911159d-bfe3-41f2-a186-01866d7b98a7\") " pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:08:29.638184 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.638154 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0911159d-bfe3-41f2-a186-01866d7b98a7-tls-cert\") pod \"authorino-5ccf5b4977-md4km\" (UID: \"0911159d-bfe3-41f2-a186-01866d7b98a7\") " pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:08:29.638184 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.638186 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9ggf\" (UniqueName: \"kubernetes.io/projected/0911159d-bfe3-41f2-a186-01866d7b98a7-kube-api-access-p9ggf\") pod \"authorino-5ccf5b4977-md4km\" (UID: \"0911159d-bfe3-41f2-a186-01866d7b98a7\") " pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:08:29.640671 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.640646 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0911159d-bfe3-41f2-a186-01866d7b98a7-tls-cert\") pod \"authorino-5ccf5b4977-md4km\" (UID: \"0911159d-bfe3-41f2-a186-01866d7b98a7\") " pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:08:29.645609 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.645585 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9ggf\" (UniqueName: \"kubernetes.io/projected/0911159d-bfe3-41f2-a186-01866d7b98a7-kube-api-access-p9ggf\") pod \"authorino-5ccf5b4977-md4km\" (UID: \"0911159d-bfe3-41f2-a186-01866d7b98a7\") " pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:08:29.709334 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.709305 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:08:29.845271 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.845232 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ccf5b4977-md4km"] Jun 13 01:08:29.847501 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:08:29.847477 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0911159d_bfe3_41f2_a186_01866d7b98a7.slice/crio-c0fee9dce312c688e8e996fa3c09694d50a29466ae3dd484d87c39024ccd37ee WatchSource:0}: Error finding container c0fee9dce312c688e8e996fa3c09694d50a29466ae3dd484d87c39024ccd37ee: Status 404 returned error can't find the container with id c0fee9dce312c688e8e996fa3c09694d50a29466ae3dd484d87c39024ccd37ee Jun 13 01:08:29.930952 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.930916 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccf5b4977-md4km" event={"ID":"0911159d-bfe3-41f2-a186-01866d7b98a7","Type":"ContainerStarted","Data":"c0fee9dce312c688e8e996fa3c09694d50a29466ae3dd484d87c39024ccd37ee"} Jun 13 01:08:29.931100 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.930966 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-784b644797-cwkv8" Jun 13 01:08:29.931100 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.930966 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-87k2f" Jun 13 01:08:29.937579 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.937560 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-784b644797-cwkv8" Jun 13 01:08:29.940296 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:29.940279 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-87k2f" Jun 13 01:08:30.042502 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.042471 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvsfq\" (UniqueName: \"kubernetes.io/projected/55679c68-0cfc-4513-8bac-15dbb323dbbe-kube-api-access-qvsfq\") pod \"55679c68-0cfc-4513-8bac-15dbb323dbbe\" (UID: \"55679c68-0cfc-4513-8bac-15dbb323dbbe\") " Jun 13 01:08:30.042659 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.042511 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcs6r\" (UniqueName: \"kubernetes.io/projected/901be467-bb07-4810-90b7-09169fee8905-kube-api-access-xcs6r\") pod \"901be467-bb07-4810-90b7-09169fee8905\" (UID: \"901be467-bb07-4810-90b7-09169fee8905\") " Jun 13 01:08:30.044524 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.044460 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/901be467-bb07-4810-90b7-09169fee8905-kube-api-access-xcs6r" (OuterVolumeSpecName: "kube-api-access-xcs6r") pod "901be467-bb07-4810-90b7-09169fee8905" (UID: "901be467-bb07-4810-90b7-09169fee8905"). InnerVolumeSpecName "kube-api-access-xcs6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:08:30.044524 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.044514 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55679c68-0cfc-4513-8bac-15dbb323dbbe-kube-api-access-qvsfq" (OuterVolumeSpecName: "kube-api-access-qvsfq") pod "55679c68-0cfc-4513-8bac-15dbb323dbbe" (UID: "55679c68-0cfc-4513-8bac-15dbb323dbbe"). InnerVolumeSpecName "kube-api-access-qvsfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:08:30.143322 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.143271 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvsfq\" (UniqueName: \"kubernetes.io/projected/55679c68-0cfc-4513-8bac-15dbb323dbbe-kube-api-access-qvsfq\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:08:30.143322 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.143314 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcs6r\" (UniqueName: \"kubernetes.io/projected/901be467-bb07-4810-90b7-09169fee8905-kube-api-access-xcs6r\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:08:30.937336 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.937301 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccf5b4977-md4km" event={"ID":"0911159d-bfe3-41f2-a186-01866d7b98a7","Type":"ContainerStarted","Data":"31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c"} Jun 13 01:08:30.937336 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.937328 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-87k2f" Jun 13 01:08:30.937530 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.937497 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-784b644797-cwkv8" Jun 13 01:08:30.951881 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.951834 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5ccf5b4977-md4km" podStartSLOduration=1.606237253 podStartE2EDuration="1.951821191s" totalImagesPullingTime="345.583938ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:08:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:08:30.951245962 +0000 UTC m=+750.525174850" watchObservedRunningTime="2026-06-13 01:08:30.951821191 +0000 UTC m=+750.525750078" Jun 13 01:08:30.979360 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.979326 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tlgsb"] Jun 13 01:08:30.979991 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.979522 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-tlgsb" podUID="eea04aeb-7d42-49af-b066-ce0d4c6ee271" containerName="authorino" containerID="cri-o://bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6" gracePeriod=30 Jun 13 01:08:30.985442 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.985408 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-87k2f"] Jun 13 01:08:30.992670 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:30.992640 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-87k2f"] Jun 13 01:08:31.012407 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.012383 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-784b644797-cwkv8"] Jun 13 01:08:31.014535 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.014508 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-784b644797-cwkv8"] Jun 13 01:08:31.229820 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.229794 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tlgsb" Jun 13 01:08:31.355446 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.355421 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drfzv\" (UniqueName: \"kubernetes.io/projected/eea04aeb-7d42-49af-b066-ce0d4c6ee271-kube-api-access-drfzv\") pod \"eea04aeb-7d42-49af-b066-ce0d4c6ee271\" (UID: \"eea04aeb-7d42-49af-b066-ce0d4c6ee271\") " Jun 13 01:08:31.357644 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.357614 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea04aeb-7d42-49af-b066-ce0d4c6ee271-kube-api-access-drfzv" (OuterVolumeSpecName: "kube-api-access-drfzv") pod "eea04aeb-7d42-49af-b066-ce0d4c6ee271" (UID: "eea04aeb-7d42-49af-b066-ce0d4c6ee271"). InnerVolumeSpecName "kube-api-access-drfzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:08:31.456132 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.456055 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drfzv\" (UniqueName: \"kubernetes.io/projected/eea04aeb-7d42-49af-b066-ce0d4c6ee271-kube-api-access-drfzv\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:08:31.943291 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.943255 2552 generic.go:356] "Generic (PLEG): container finished" podID="eea04aeb-7d42-49af-b066-ce0d4c6ee271" containerID="bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6" exitCode=0 Jun 13 01:08:31.943482 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.943301 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tlgsb" Jun 13 01:08:31.943482 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.943330 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tlgsb" event={"ID":"eea04aeb-7d42-49af-b066-ce0d4c6ee271","Type":"ContainerDied","Data":"bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6"} Jun 13 01:08:31.943482 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.943373 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tlgsb" event={"ID":"eea04aeb-7d42-49af-b066-ce0d4c6ee271","Type":"ContainerDied","Data":"4264ac4a04ed9901bc0292dd2e7ee7332174e510da4b14a28ba7fe26133ea74a"} Jun 13 01:08:31.943482 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.943390 2552 scope.go:117] "RemoveContainer" containerID="bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6" Jun 13 01:08:31.952504 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.952486 2552 scope.go:117] "RemoveContainer" containerID="bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6" Jun 13 01:08:31.952826 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:08:31.952773 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6\": container with ID starting with bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6 not found: ID does not exist" containerID="bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6" Jun 13 01:08:31.952891 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.952835 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6"} err="failed to get container status \"bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6\": rpc error: code = NotFound desc = could not find container \"bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6\": container with ID starting with bafbd414ef10a4090f0f717b91fc8dceec4e5f2f7d24e7ab4c0799fbc6f7c6f6 not found: ID does not exist" Jun 13 01:08:31.963271 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.963249 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tlgsb"] Jun 13 01:08:31.969969 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:31.969950 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tlgsb"] Jun 13 01:08:32.855946 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:32.855917 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55679c68-0cfc-4513-8bac-15dbb323dbbe" path="/var/lib/kubelet/pods/55679c68-0cfc-4513-8bac-15dbb323dbbe/volumes" Jun 13 01:08:32.856305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:32.856150 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="901be467-bb07-4810-90b7-09169fee8905" path="/var/lib/kubelet/pods/901be467-bb07-4810-90b7-09169fee8905/volumes" Jun 13 01:08:32.856360 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:08:32.856348 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eea04aeb-7d42-49af-b066-ce0d4c6ee271" path="/var/lib/kubelet/pods/eea04aeb-7d42-49af-b066-ce0d4c6ee271/volumes" Jun 13 01:09:09.073721 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.073689 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-cvl8j"] Jun 13 01:09:09.074142 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.074059 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eea04aeb-7d42-49af-b066-ce0d4c6ee271" containerName="authorino" Jun 13 01:09:09.074142 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.074070 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="eea04aeb-7d42-49af-b066-ce0d4c6ee271" containerName="authorino" Jun 13 01:09:09.074142 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.074138 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="eea04aeb-7d42-49af-b066-ce0d4c6ee271" containerName="authorino" Jun 13 01:09:09.078563 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.078543 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" Jun 13 01:09:09.081065 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.081045 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 13 01:09:09.081166 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.081048 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-mtv98\"" Jun 13 01:09:09.089450 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.089422 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-cvl8j"] Jun 13 01:09:09.112851 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.112830 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-8c5m4"] Jun 13 01:09:09.116236 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.116218 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" Jun 13 01:09:09.124399 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.124376 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-8c5m4"] Jun 13 01:09:09.173305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.173286 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blxmx\" (UniqueName: \"kubernetes.io/projected/90eb8af4-d86b-42d2-836d-85b8c4d69029-kube-api-access-blxmx\") pod \"payload-processing-699b6cd6c-cvl8j\" (UID: \"90eb8af4-d86b-42d2-836d-85b8c4d69029\") " pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" Jun 13 01:09:09.173409 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.173350 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzwdp\" (UniqueName: \"kubernetes.io/projected/20f5b379-eeb8-4c02-9e97-70d8c7d29792-kube-api-access-rzwdp\") pod \"payload-pre-processing-579785bc64-8c5m4\" (UID: \"20f5b379-eeb8-4c02-9e97-70d8c7d29792\") " pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" Jun 13 01:09:09.274603 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.274570 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blxmx\" (UniqueName: \"kubernetes.io/projected/90eb8af4-d86b-42d2-836d-85b8c4d69029-kube-api-access-blxmx\") pod \"payload-processing-699b6cd6c-cvl8j\" (UID: \"90eb8af4-d86b-42d2-836d-85b8c4d69029\") " pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" Jun 13 01:09:09.274754 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.274708 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzwdp\" (UniqueName: \"kubernetes.io/projected/20f5b379-eeb8-4c02-9e97-70d8c7d29792-kube-api-access-rzwdp\") pod \"payload-pre-processing-579785bc64-8c5m4\" (UID: \"20f5b379-eeb8-4c02-9e97-70d8c7d29792\") " pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" Jun 13 01:09:09.282670 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.282641 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blxmx\" (UniqueName: \"kubernetes.io/projected/90eb8af4-d86b-42d2-836d-85b8c4d69029-kube-api-access-blxmx\") pod \"payload-processing-699b6cd6c-cvl8j\" (UID: \"90eb8af4-d86b-42d2-836d-85b8c4d69029\") " pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" Jun 13 01:09:09.282833 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.282725 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzwdp\" (UniqueName: \"kubernetes.io/projected/20f5b379-eeb8-4c02-9e97-70d8c7d29792-kube-api-access-rzwdp\") pod \"payload-pre-processing-579785bc64-8c5m4\" (UID: \"20f5b379-eeb8-4c02-9e97-70d8c7d29792\") " pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" Jun 13 01:09:09.390457 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.390377 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" Jun 13 01:09:09.426762 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.426733 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" Jun 13 01:09:09.557183 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.557098 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-cvl8j"] Jun 13 01:09:09.572806 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:09:09.564704 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90eb8af4_d86b_42d2_836d_85b8c4d69029.slice/crio-027ab6b0dbabbbb2c027b294a968858471cfb98291bcf04eb233c98f927b1627 WatchSource:0}: Error finding container 027ab6b0dbabbbb2c027b294a968858471cfb98291bcf04eb233c98f927b1627: Status 404 returned error can't find the container with id 027ab6b0dbabbbb2c027b294a968858471cfb98291bcf04eb233c98f927b1627 Jun 13 01:09:09.632551 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:09.632526 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-8c5m4"] Jun 13 01:09:09.634592 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:09:09.634565 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f5b379_eeb8_4c02_9e97_70d8c7d29792.slice/crio-089131a112663a4167dfc7e6f66f1d50eb413fa6bd3d1b6cbac4011486203b8d WatchSource:0}: Error finding container 089131a112663a4167dfc7e6f66f1d50eb413fa6bd3d1b6cbac4011486203b8d: Status 404 returned error can't find the container with id 089131a112663a4167dfc7e6f66f1d50eb413fa6bd3d1b6cbac4011486203b8d Jun 13 01:09:10.128002 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:10.127914 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" event={"ID":"20f5b379-eeb8-4c02-9e97-70d8c7d29792","Type":"ContainerStarted","Data":"089131a112663a4167dfc7e6f66f1d50eb413fa6bd3d1b6cbac4011486203b8d"} Jun 13 01:09:10.130043 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:10.129985 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" event={"ID":"90eb8af4-d86b-42d2-836d-85b8c4d69029","Type":"ContainerStarted","Data":"027ab6b0dbabbbb2c027b294a968858471cfb98291bcf04eb233c98f927b1627"} Jun 13 01:09:13.146211 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:13.146171 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" event={"ID":"20f5b379-eeb8-4c02-9e97-70d8c7d29792","Type":"ContainerStarted","Data":"2fe26d25a1a1d6c8fd7e006eda1fe6dd8f8901c07162ac32494fb71cf1a2c797"} Jun 13 01:09:13.146691 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:13.146235 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" Jun 13 01:09:13.147857 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:13.147828 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" event={"ID":"90eb8af4-d86b-42d2-836d-85b8c4d69029","Type":"ContainerStarted","Data":"f21b04321c3b0960fabcb62a48d205140eecc0d2a650d24dee57ed62ae42676d"} Jun 13 01:09:13.147996 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:13.147951 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" Jun 13 01:09:13.162187 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:13.162138 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" podStartSLOduration=1.357620706 podStartE2EDuration="4.16212214s" totalImagesPullingTime="2.804501434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:09:13.161286448 +0000 UTC m=+792.735215336" watchObservedRunningTime="2026-06-13 01:09:13.16212214 +0000 UTC m=+792.736051047" Jun 13 01:09:13.181264 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:13.181209 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" podStartSLOduration=1.324629122 podStartE2EDuration="4.181191778s" totalImagesPullingTime="2.856562656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:09:13.177566056 +0000 UTC m=+792.751494972" watchObservedRunningTime="2026-06-13 01:09:13.181191778 +0000 UTC m=+792.755120669" Jun 13 01:09:17.170613 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:17.170580 2552 generic.go:356] "Generic (PLEG): container finished" podID="f4876ece-d3c1-45a5-9c59-300d782c7bdc" containerID="7a4cb3191b9a3c9084aeb15a749804d6b2c1b6ebb3e5e00928cb050e9d419b5c" exitCode=0 Jun 13 01:09:17.171052 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:17.170621 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" event={"ID":"f4876ece-d3c1-45a5-9c59-300d782c7bdc","Type":"ContainerDied","Data":"7a4cb3191b9a3c9084aeb15a749804d6b2c1b6ebb3e5e00928cb050e9d419b5c"} Jun 13 01:09:17.171052 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:17.171032 2552 scope.go:117] "RemoveContainer" containerID="7a4cb3191b9a3c9084aeb15a749804d6b2c1b6ebb3e5e00928cb050e9d419b5c" Jun 13 01:09:17.172051 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:17.172016 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 13 01:09:17.172163 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:17.172105 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 13 01:09:17.172163 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:17.172153 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 13 01:09:18.180626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:18.180585 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" event={"ID":"f4876ece-d3c1-45a5-9c59-300d782c7bdc","Type":"ContainerStarted","Data":"ef4f183b8c0d9c54400823c587684a9e5319f90f8f877addea4339351e26f7a6"} Jun 13 01:09:18.458478 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:18.458383 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:09:18.981051 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:18.981014 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 13 01:09:18.981282 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:18.981251 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" containerID="cri-o://c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500" gracePeriod=30 Jun 13 01:09:19.077026 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:19.076932 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:09:19.078215 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:19.078186 2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-pdnf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jun 13 01:09:19.078340 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:19.078242 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" podUID="f4876ece-d3c1-45a5-9c59-300d782c7bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jun 13 01:09:20.076349 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:20.076311 2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-pdnf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jun 13 01:09:20.076964 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:20.076932 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" podUID="f4876ece-d3c1-45a5-9c59-300d782c7bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jun 13 01:09:21.056031 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.056008 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.095297 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.095267 2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-pdnf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jun 13 01:09:21.095749 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.095319 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" podUID="f4876ece-d3c1-45a5-9c59-300d782c7bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jun 13 01:09:21.195490 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.195406 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8gcw\" (UniqueName: \"kubernetes.io/projected/69bca32c-6609-4411-b8f5-03099d6f743b-kube-api-access-r8gcw\") pod \"69bca32c-6609-4411-b8f5-03099d6f743b\" (UID: \"69bca32c-6609-4411-b8f5-03099d6f743b\") " Jun 13 01:09:21.197679 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.197651 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69bca32c-6609-4411-b8f5-03099d6f743b-kube-api-access-r8gcw" (OuterVolumeSpecName: "kube-api-access-r8gcw") pod "69bca32c-6609-4411-b8f5-03099d6f743b" (UID: "69bca32c-6609-4411-b8f5-03099d6f743b"). InnerVolumeSpecName "kube-api-access-r8gcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:09:21.200454 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.200424 2552 generic.go:356] "Generic (PLEG): container finished" podID="69bca32c-6609-4411-b8f5-03099d6f743b" containerID="c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500" exitCode=143 Jun 13 01:09:21.200454 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.200462 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.200640 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.200497 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"69bca32c-6609-4411-b8f5-03099d6f743b","Type":"ContainerDied","Data":"c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500"} Jun 13 01:09:21.200640 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.200551 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"69bca32c-6609-4411-b8f5-03099d6f743b","Type":"ContainerDied","Data":"7dd842f23b0f8da410be26bf2c5138204d0c7d2c85cfab62a923247937bfd66d"} Jun 13 01:09:21.200640 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.200573 2552 scope.go:117] "RemoveContainer" containerID="c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500" Jun 13 01:09:21.211750 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.211735 2552 scope.go:117] "RemoveContainer" containerID="c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500" Jun 13 01:09:21.212051 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:09:21.212029 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500\": container with ID starting with c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500 not found: ID does not exist" containerID="c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500" Jun 13 01:09:21.212120 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.212063 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500"} err="failed to get container status \"c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500\": rpc error: code = NotFound desc = could not find container \"c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500\": container with ID starting with c102a8c9d172eba4fffd242c24c230a73cbeed92f9ffd982a2ea9ad5f65b1500 not found: ID does not exist" Jun 13 01:09:21.221591 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.221569 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 13 01:09:21.225164 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.225145 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 13 01:09:21.246719 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.246695 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 13 01:09:21.247096 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.247076 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" Jun 13 01:09:21.247096 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.247095 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" Jun 13 01:09:21.247276 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.247201 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" containerName="keycloak" Jun 13 01:09:21.252662 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.252643 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.255158 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.255141 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 13 01:09:21.255245 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.255159 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 13 01:09:21.255245 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.255179 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-wzf59\"" Jun 13 01:09:21.255245 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.255164 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 13 01:09:21.255245 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.255221 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 13 01:09:21.258493 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.258473 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 13 01:09:21.296155 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.296132 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8gcw\" (UniqueName: \"kubernetes.io/projected/69bca32c-6609-4411-b8f5-03099d6f743b-kube-api-access-r8gcw\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:09:21.396491 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.396464 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxhjz\" (UniqueName: \"kubernetes.io/projected/551708a2-c53d-4fa2-915c-f21cf46b4f8d-kube-api-access-sxhjz\") pod \"maas-keycloak-0\" (UID: \"551708a2-c53d-4fa2-915c-f21cf46b4f8d\") " pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.396636 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.396516 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/551708a2-c53d-4fa2-915c-f21cf46b4f8d-test-realms\") pod \"maas-keycloak-0\" (UID: \"551708a2-c53d-4fa2-915c-f21cf46b4f8d\") " pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.497311 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.497234 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/551708a2-c53d-4fa2-915c-f21cf46b4f8d-test-realms\") pod \"maas-keycloak-0\" (UID: \"551708a2-c53d-4fa2-915c-f21cf46b4f8d\") " pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.497311 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.497306 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxhjz\" (UniqueName: \"kubernetes.io/projected/551708a2-c53d-4fa2-915c-f21cf46b4f8d-kube-api-access-sxhjz\") pod \"maas-keycloak-0\" (UID: \"551708a2-c53d-4fa2-915c-f21cf46b4f8d\") " pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.497910 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.497891 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/551708a2-c53d-4fa2-915c-f21cf46b4f8d-test-realms\") pod \"maas-keycloak-0\" (UID: \"551708a2-c53d-4fa2-915c-f21cf46b4f8d\") " pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.505338 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.505315 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxhjz\" (UniqueName: \"kubernetes.io/projected/551708a2-c53d-4fa2-915c-f21cf46b4f8d-kube-api-access-sxhjz\") pod \"maas-keycloak-0\" (UID: \"551708a2-c53d-4fa2-915c-f21cf46b4f8d\") " pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.563119 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.563097 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:21.730601 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:21.730578 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 13 01:09:21.732493 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:09:21.732465 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551708a2_c53d_4fa2_915c_f21cf46b4f8d.slice/crio-ec8674678d51e8807021538cc589360cf5ba37db6478e71e58538b80ed8fa0cf WatchSource:0}: Error finding container ec8674678d51e8807021538cc589360cf5ba37db6478e71e58538b80ed8fa0cf: Status 404 returned error can't find the container with id ec8674678d51e8807021538cc589360cf5ba37db6478e71e58538b80ed8fa0cf Jun 13 01:09:22.079959 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:22.079938 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:09:22.080790 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:22.080754 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-pdnf7" Jun 13 01:09:22.208692 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:22.208659 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"551708a2-c53d-4fa2-915c-f21cf46b4f8d","Type":"ContainerStarted","Data":"331e2e3b6f512194d1f2eba7c5b99630bd871f3273d9bfa512e4e700ac4087cd"} Jun 13 01:09:22.209179 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:22.208716 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"551708a2-c53d-4fa2-915c-f21cf46b4f8d","Type":"ContainerStarted","Data":"ec8674678d51e8807021538cc589360cf5ba37db6478e71e58538b80ed8fa0cf"} Jun 13 01:09:22.227299 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:22.227243 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.898124377 podStartE2EDuration="1.227226822s" totalImagesPullingTime="329.102445ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:09:22.226678622 +0000 UTC m=+801.800607527" watchObservedRunningTime="2026-06-13 01:09:22.227226822 +0000 UTC m=+801.801155721" Jun 13 01:09:22.563865 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:22.563827 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:22.565821 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:22.565757 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:22.858458 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:22.858349 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69bca32c-6609-4411-b8f5-03099d6f743b" path="/var/lib/kubelet/pods/69bca32c-6609-4411-b8f5-03099d6f743b/volumes" Jun 13 01:09:23.564397 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:23.564347 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:24.156418 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:24.156386 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-cvl8j" Jun 13 01:09:24.156775 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:24.156757 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-8c5m4" Jun 13 01:09:24.564257 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:24.564212 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:25.563619 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:25.563572 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:26.564313 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:26.564266 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:27.563913 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:27.563849 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:28.563598 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:28.563546 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:29.564619 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:29.564572 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:30.564402 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:30.564356 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:31.563290 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:31.563254 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:31.563837 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:31.563572 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:32.564382 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:32.564328 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:33.564550 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:33.564492 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:34.564357 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:34.564308 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.47:9000/health/started\": dial tcp 10.132.0.47:9000: connect: connection refused" Jun 13 01:09:35.687179 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:35.687121 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:35.704939 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:35.704888 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="551708a2-c53d-4fa2-915c-f21cf46b4f8d" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 13 01:09:45.694047 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:45.694014 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 13 01:09:58.042085 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.042050 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ccf5b4977-md4km"] Jun 13 01:09:58.042521 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.042218 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5ccf5b4977-md4km" podUID="0911159d-bfe3-41f2-a186-01866d7b98a7" containerName="authorino" containerID="cri-o://31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c" gracePeriod=30 Jun 13 01:09:58.301452 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.301426 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:09:58.418116 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.418085 2552 generic.go:356] "Generic (PLEG): container finished" podID="0911159d-bfe3-41f2-a186-01866d7b98a7" containerID="31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c" exitCode=0 Jun 13 01:09:58.418252 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.418125 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccf5b4977-md4km" Jun 13 01:09:58.418252 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.418134 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccf5b4977-md4km" event={"ID":"0911159d-bfe3-41f2-a186-01866d7b98a7","Type":"ContainerDied","Data":"31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c"} Jun 13 01:09:58.418252 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.418160 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccf5b4977-md4km" event={"ID":"0911159d-bfe3-41f2-a186-01866d7b98a7","Type":"ContainerDied","Data":"c0fee9dce312c688e8e996fa3c09694d50a29466ae3dd484d87c39024ccd37ee"} Jun 13 01:09:58.418252 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.418177 2552 scope.go:117] "RemoveContainer" containerID="31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c" Jun 13 01:09:58.427392 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.427375 2552 scope.go:117] "RemoveContainer" containerID="31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c" Jun 13 01:09:58.427634 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:09:58.427615 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c\": container with ID starting with 31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c not found: ID does not exist" containerID="31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c" Jun 13 01:09:58.427699 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.427647 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c"} err="failed to get container status \"31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c\": rpc error: code = NotFound desc = could not find container \"31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c\": container with ID starting with 31ce228a61f8f6d1cbdf92d5b30b3fbe1b18087281e5f12ad89ba6237210071c not found: ID does not exist" Jun 13 01:09:58.444026 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.444005 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9ggf\" (UniqueName: \"kubernetes.io/projected/0911159d-bfe3-41f2-a186-01866d7b98a7-kube-api-access-p9ggf\") pod \"0911159d-bfe3-41f2-a186-01866d7b98a7\" (UID: \"0911159d-bfe3-41f2-a186-01866d7b98a7\") " Jun 13 01:09:58.444117 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.444063 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0911159d-bfe3-41f2-a186-01866d7b98a7-tls-cert\") pod \"0911159d-bfe3-41f2-a186-01866d7b98a7\" (UID: \"0911159d-bfe3-41f2-a186-01866d7b98a7\") " Jun 13 01:09:58.445962 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.445938 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0911159d-bfe3-41f2-a186-01866d7b98a7-kube-api-access-p9ggf" (OuterVolumeSpecName: "kube-api-access-p9ggf") pod "0911159d-bfe3-41f2-a186-01866d7b98a7" (UID: "0911159d-bfe3-41f2-a186-01866d7b98a7"). InnerVolumeSpecName "kube-api-access-p9ggf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:09:58.454051 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.454024 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0911159d-bfe3-41f2-a186-01866d7b98a7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0911159d-bfe3-41f2-a186-01866d7b98a7" (UID: "0911159d-bfe3-41f2-a186-01866d7b98a7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 13 01:09:58.545272 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.545250 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9ggf\" (UniqueName: \"kubernetes.io/projected/0911159d-bfe3-41f2-a186-01866d7b98a7-kube-api-access-p9ggf\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:09:58.545272 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.545269 2552 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0911159d-bfe3-41f2-a186-01866d7b98a7-tls-cert\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:09:58.744748 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.741554 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ccf5b4977-md4km"] Jun 13 01:09:58.750143 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.750120 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5ccf5b4977-md4km"] Jun 13 01:09:58.854887 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:09:58.854860 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0911159d-bfe3-41f2-a186-01866d7b98a7" path="/var/lib/kubelet/pods/0911159d-bfe3-41f2-a186-01866d7b98a7/volumes" Jun 13 01:10:11.926167 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.926136 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn"] Jun 13 01:10:11.926520 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.926509 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0911159d-bfe3-41f2-a186-01866d7b98a7" containerName="authorino" Jun 13 01:10:11.926562 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.926520 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0911159d-bfe3-41f2-a186-01866d7b98a7" containerName="authorino" Jun 13 01:10:11.926613 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.926603 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="0911159d-bfe3-41f2-a186-01866d7b98a7" containerName="authorino" Jun 13 01:10:11.931920 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.931900 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:11.934974 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.934954 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sqs47\"" Jun 13 01:10:11.935112 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.934989 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 13 01:10:11.936047 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.936029 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 13 01:10:11.936164 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.936148 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 13 01:10:11.945491 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:11.945466 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn"] Jun 13 01:10:12.060054 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.060025 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.060054 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.060055 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.060206 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.060078 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.060206 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.060106 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.060206 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.060132 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.060206 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.060177 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhgx2\" (UniqueName: \"kubernetes.io/projected/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-kube-api-access-mhgx2\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.161340 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.161310 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.161481 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.161349 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.161481 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.161415 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhgx2\" (UniqueName: \"kubernetes.io/projected/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-kube-api-access-mhgx2\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.161591 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.161480 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.161591 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.161509 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.161591 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.161544 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.161757 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.161735 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.161850 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.161826 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.161938 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.161920 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.163599 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.163575 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.163879 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.163858 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.176007 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.175985 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhgx2\" (UniqueName: \"kubernetes.io/projected/2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e-kube-api-access-mhgx2\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn\" (UID: \"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.243057 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.242923 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:12.383107 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.383081 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn"] Jun 13 01:10:12.385311 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:10:12.385277 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c27fdd0_55b4_4cd7_ac5b_7bf9a85cb92e.slice/crio-deac295679f53694ad9d5ef2d412028d4a1dda36a9be32e6bbdf309f4925e062 WatchSource:0}: Error finding container deac295679f53694ad9d5ef2d412028d4a1dda36a9be32e6bbdf309f4925e062: Status 404 returned error can't find the container with id deac295679f53694ad9d5ef2d412028d4a1dda36a9be32e6bbdf309f4925e062 Jun 13 01:10:12.487948 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:12.487915 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" event={"ID":"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e","Type":"ContainerStarted","Data":"deac295679f53694ad9d5ef2d412028d4a1dda36a9be32e6bbdf309f4925e062"} Jun 13 01:10:16.513104 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:16.513065 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" event={"ID":"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e","Type":"ContainerStarted","Data":"599e75d62e6e4ddcc306fea49bc688392117069e791d83817e166fe0088ed8fb"} Jun 13 01:10:17.519504 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:17.519463 2552 generic.go:356] "Generic (PLEG): container finished" podID="2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e" containerID="599e75d62e6e4ddcc306fea49bc688392117069e791d83817e166fe0088ed8fb" exitCode=0 Jun 13 01:10:17.519926 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:17.519534 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" event={"ID":"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e","Type":"ContainerDied","Data":"599e75d62e6e4ddcc306fea49bc688392117069e791d83817e166fe0088ed8fb"} Jun 13 01:10:19.529980 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:19.529949 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" event={"ID":"2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e","Type":"ContainerStarted","Data":"b55e361afe1303c01e4c02846cc47267d4c51ffc30cca60b04c1d8ef99453261"} Jun 13 01:10:19.552871 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:19.552825 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" podStartSLOduration=3.082191446 podStartE2EDuration="8.552811716s" totalImagesPullingTime="4.279565789s" totalInitContainerRuntime="1.191054481s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:10:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:10:19.550979164 +0000 UTC m=+859.124908068" watchObservedRunningTime="2026-06-13 01:10:19.552811716 +0000 UTC m=+859.126740603" Jun 13 01:10:22.243639 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:22.243607 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:22.244222 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:22.243668 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:22.255953 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:22.255930 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:22.555070 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:22.555047 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn" Jun 13 01:10:47.705551 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.705473 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs"] Jun 13 01:10:47.709470 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.709449 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.712083 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.712059 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 13 01:10:47.719856 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.719833 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs"] Jun 13 01:10:47.752044 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.752015 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.752154 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.752059 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.752154 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.752120 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lssq5\" (UniqueName: \"kubernetes.io/projected/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-kube-api-access-lssq5\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.752264 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.752186 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.752264 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.752222 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.752367 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.752290 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.853310 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.853285 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lssq5\" (UniqueName: \"kubernetes.io/projected/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-kube-api-access-lssq5\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.853409 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.853326 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.853409 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.853368 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.853523 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.853417 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.853523 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.853464 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.853523 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.853487 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.853707 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.853688 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.853838 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.853817 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.853925 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.853889 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.855841 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.855823 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.855960 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.855940 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:47.861259 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:47.861233 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lssq5\" (UniqueName: \"kubernetes.io/projected/7c9d6f93-c84f-4bff-8050-4ea64cdd328c-kube-api-access-lssq5\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs\" (UID: \"7c9d6f93-c84f-4bff-8050-4ea64cdd328c\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:48.037875 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:48.037813 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:48.172263 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:48.172220 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs"] Jun 13 01:10:48.174771 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:10:48.174744 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9d6f93_c84f_4bff_8050_4ea64cdd328c.slice/crio-5bf9d7fcc96ad3118d9689757186217dc7c1406f16082a0a50a9a8d663bae24d WatchSource:0}: Error finding container 5bf9d7fcc96ad3118d9689757186217dc7c1406f16082a0a50a9a8d663bae24d: Status 404 returned error can't find the container with id 5bf9d7fcc96ad3118d9689757186217dc7c1406f16082a0a50a9a8d663bae24d Jun 13 01:10:48.676390 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:48.676355 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" event={"ID":"7c9d6f93-c84f-4bff-8050-4ea64cdd328c","Type":"ContainerStarted","Data":"a979a428142b850027bfa4062c4243ceafe41d4cd13ffd6b761e0a3f5029a317"} Jun 13 01:10:48.676557 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:48.676405 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" event={"ID":"7c9d6f93-c84f-4bff-8050-4ea64cdd328c","Type":"ContainerStarted","Data":"5bf9d7fcc96ad3118d9689757186217dc7c1406f16082a0a50a9a8d663bae24d"} Jun 13 01:10:51.693243 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:51.693208 2552 generic.go:356] "Generic (PLEG): container finished" podID="7c9d6f93-c84f-4bff-8050-4ea64cdd328c" containerID="a979a428142b850027bfa4062c4243ceafe41d4cd13ffd6b761e0a3f5029a317" exitCode=0 Jun 13 01:10:51.693612 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:51.693280 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" event={"ID":"7c9d6f93-c84f-4bff-8050-4ea64cdd328c","Type":"ContainerDied","Data":"a979a428142b850027bfa4062c4243ceafe41d4cd13ffd6b761e0a3f5029a317"} Jun 13 01:10:52.700676 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:52.700646 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" event={"ID":"7c9d6f93-c84f-4bff-8050-4ea64cdd328c","Type":"ContainerStarted","Data":"cb0b730a44ce1ee0202358546a926e382834049c3cf9470dbaa70d6ef4d0593d"} Jun 13 01:10:52.719655 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:52.719605 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" podStartSLOduration=2.718367824 podStartE2EDuration="5.71958645s" totalImagesPullingTime="193.045602ms" totalInitContainerRuntime="2.808173024s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:10:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:10:52.718551235 +0000 UTC m=+892.292480123" watchObservedRunningTime="2026-06-13 01:10:52.71958645 +0000 UTC m=+892.293515340" Jun 13 01:10:58.038250 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:58.038219 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:58.038828 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:58.038406 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:58.051029 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:58.051005 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:10:58.741315 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:10:58.741281 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs" Jun 13 01:11:00.878082 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:00.878045 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:11:00.880626 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:00.880603 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:11:11.720447 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.720416 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln"] Jun 13 01:11:11.725836 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.725819 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.729450 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.729430 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 13 01:11:11.739259 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.739238 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln"] Jun 13 01:11:11.864150 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.864124 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.864150 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.864151 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48efb87-ef25-4c57-b84e-b63b2293c21f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.864383 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.864178 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.864383 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.864227 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.864383 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.864326 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.864383 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.864375 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv5kd\" (UniqueName: \"kubernetes.io/projected/f48efb87-ef25-4c57-b84e-b63b2293c21f-kube-api-access-fv5kd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.965834 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.965791 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.965834 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.965835 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.966073 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.965907 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.966073 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.965944 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv5kd\" (UniqueName: \"kubernetes.io/projected/f48efb87-ef25-4c57-b84e-b63b2293c21f-kube-api-access-fv5kd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.966073 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.966035 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.966073 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.966062 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48efb87-ef25-4c57-b84e-b63b2293c21f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.966538 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.966285 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.966538 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.966448 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.966843 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.966820 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.968699 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.968677 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48efb87-ef25-4c57-b84e-b63b2293c21f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.968699 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.968691 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48efb87-ef25-4c57-b84e-b63b2293c21f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:11.974970 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:11.974910 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv5kd\" (UniqueName: \"kubernetes.io/projected/f48efb87-ef25-4c57-b84e-b63b2293c21f-kube-api-access-fv5kd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln\" (UID: \"f48efb87-ef25-4c57-b84e-b63b2293c21f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:12.036680 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:12.036656 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:12.177663 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:12.177635 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln"] Jun 13 01:11:12.179853 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:11:12.179824 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf48efb87_ef25_4c57_b84e_b63b2293c21f.slice/crio-0766248df7d50a456dfba3ee0960489da9485dfaf846e03694985d7a3cbd943c WatchSource:0}: Error finding container 0766248df7d50a456dfba3ee0960489da9485dfaf846e03694985d7a3cbd943c: Status 404 returned error can't find the container with id 0766248df7d50a456dfba3ee0960489da9485dfaf846e03694985d7a3cbd943c Jun 13 01:11:12.797460 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:12.797426 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" event={"ID":"f48efb87-ef25-4c57-b84e-b63b2293c21f","Type":"ContainerStarted","Data":"8d76865d26831ac317f61a139ef8a386df784204b7152d06d09a3946fc3a9e9b"} Jun 13 01:11:12.797902 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:12.797527 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" event={"ID":"f48efb87-ef25-4c57-b84e-b63b2293c21f","Type":"ContainerStarted","Data":"0766248df7d50a456dfba3ee0960489da9485dfaf846e03694985d7a3cbd943c"} Jun 13 01:11:13.804308 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:13.804271 2552 generic.go:356] "Generic (PLEG): container finished" podID="f48efb87-ef25-4c57-b84e-b63b2293c21f" containerID="8d76865d26831ac317f61a139ef8a386df784204b7152d06d09a3946fc3a9e9b" exitCode=0 Jun 13 01:11:13.804749 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:13.804312 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" event={"ID":"f48efb87-ef25-4c57-b84e-b63b2293c21f","Type":"ContainerDied","Data":"8d76865d26831ac317f61a139ef8a386df784204b7152d06d09a3946fc3a9e9b"} Jun 13 01:11:14.811050 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:14.811015 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" event={"ID":"f48efb87-ef25-4c57-b84e-b63b2293c21f","Type":"ContainerStarted","Data":"f1105fe480d8d6d5b2f6f8e3ba930b987aaffa722909b7a3558631b7fe54f35a"} Jun 13 01:11:14.831828 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:14.831766 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" podStartSLOduration=2.341744334 podStartE2EDuration="3.831751175s" totalImagesPullingTime="175.4197ms" totalInitContainerRuntime="1.314587141s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:11:14.829799997 +0000 UTC m=+914.403728876" watchObservedRunningTime="2026-06-13 01:11:14.831751175 +0000 UTC m=+914.405680063" Jun 13 01:11:22.037562 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:22.037518 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:22.037976 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:22.037608 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:22.049953 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:22.049930 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:22.859314 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:22.859288 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln" Jun 13 01:11:24.250250 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.250215 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj"] Jun 13 01:11:24.254656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.254634 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.261717 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.261043 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 13 01:11:24.282383 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.282351 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj"] Jun 13 01:11:24.371014 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.370950 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa08828e-98fc-4431-9c34-d35e52670859-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.371014 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.370993 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.371359 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.371066 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.371359 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.371094 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.371359 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.371150 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.371359 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.371214 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhwzd\" (UniqueName: \"kubernetes.io/projected/fa08828e-98fc-4431-9c34-d35e52670859-kube-api-access-vhwzd\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.472226 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.472195 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.472408 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.472250 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhwzd\" (UniqueName: \"kubernetes.io/projected/fa08828e-98fc-4431-9c34-d35e52670859-kube-api-access-vhwzd\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.472408 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.472370 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa08828e-98fc-4431-9c34-d35e52670859-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.472491 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.472417 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.472546 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.472483 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.472546 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.472539 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.472673 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.472653 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.472904 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.472879 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.473007 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.472919 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.474606 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.474579 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa08828e-98fc-4431-9c34-d35e52670859-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.475022 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.475004 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa08828e-98fc-4431-9c34-d35e52670859-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.480633 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.480612 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhwzd\" (UniqueName: \"kubernetes.io/projected/fa08828e-98fc-4431-9c34-d35e52670859-kube-api-access-vhwzd\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj\" (UID: \"fa08828e-98fc-4431-9c34-d35e52670859\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.566393 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.566360 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:24.708814 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.708769 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj"] Jun 13 01:11:24.710988 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:11:24.710957 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa08828e_98fc_4431_9c34_d35e52670859.slice/crio-2ae5efe50c2709467ed6bd59668e87ae7efb8a68ff002c7978a825ff8022e3a1 WatchSource:0}: Error finding container 2ae5efe50c2709467ed6bd59668e87ae7efb8a68ff002c7978a825ff8022e3a1: Status 404 returned error can't find the container with id 2ae5efe50c2709467ed6bd59668e87ae7efb8a68ff002c7978a825ff8022e3a1 Jun 13 01:11:24.859424 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.859332 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" event={"ID":"fa08828e-98fc-4431-9c34-d35e52670859","Type":"ContainerStarted","Data":"dd0e36205c39020f3ccc1e377595301c336149c06e4a602e38e8c18254e9c9df"} Jun 13 01:11:24.859424 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:24.859382 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" event={"ID":"fa08828e-98fc-4431-9c34-d35e52670859","Type":"ContainerStarted","Data":"2ae5efe50c2709467ed6bd59668e87ae7efb8a68ff002c7978a825ff8022e3a1"} Jun 13 01:11:26.872173 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:26.872138 2552 generic.go:356] "Generic (PLEG): container finished" podID="fa08828e-98fc-4431-9c34-d35e52670859" containerID="dd0e36205c39020f3ccc1e377595301c336149c06e4a602e38e8c18254e9c9df" exitCode=0 Jun 13 01:11:26.872572 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:26.872193 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" event={"ID":"fa08828e-98fc-4431-9c34-d35e52670859","Type":"ContainerDied","Data":"dd0e36205c39020f3ccc1e377595301c336149c06e4a602e38e8c18254e9c9df"} Jun 13 01:11:27.879429 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:27.879395 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" event={"ID":"fa08828e-98fc-4431-9c34-d35e52670859","Type":"ContainerStarted","Data":"d46ddfdae4f74200ad9e60cc2924360c0b68ffa344b05e06b41bd73ec52d8a9b"} Jun 13 01:11:27.899955 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:27.899898 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" podStartSLOduration=2.462514511 podStartE2EDuration="3.899883575s" totalImagesPullingTime="183.079106ms" totalInitContainerRuntime="1.254289958s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:11:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:11:27.899572423 +0000 UTC m=+927.473501310" watchObservedRunningTime="2026-06-13 01:11:27.899883575 +0000 UTC m=+927.473812475" Jun 13 01:11:34.567522 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:34.567474 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:34.568154 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:34.567551 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:34.581330 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:34.581307 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:11:34.932035 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:11:34.931958 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj" Jun 13 01:16:00.910228 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:16:00.910201 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:16:00.913077 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:16:00.913053 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:21:00.942036 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:21:00.942006 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:21:00.945536 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:21:00.945511 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:24:25.882426 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:25.882385 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9"] Jun 13 01:24:25.883047 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:25.882666 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" podUID="3b991d83-6dd3-4b34-bdda-e379d7dce2eb" containerName="manager" containerID="cri-o://aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378" gracePeriod=10 Jun 13 01:24:26.151052 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.151028 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:24:26.229373 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.229341 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-extensions-socket-volume\") pod \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\" (UID: \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\") " Jun 13 01:24:26.229534 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.229381 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-698dq\" (UniqueName: \"kubernetes.io/projected/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-kube-api-access-698dq\") pod \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\" (UID: \"3b991d83-6dd3-4b34-bdda-e379d7dce2eb\") " Jun 13 01:24:26.229703 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.229682 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3b991d83-6dd3-4b34-bdda-e379d7dce2eb" (UID: "3b991d83-6dd3-4b34-bdda-e379d7dce2eb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 13 01:24:26.231381 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.231355 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-kube-api-access-698dq" (OuterVolumeSpecName: "kube-api-access-698dq") pod "3b991d83-6dd3-4b34-bdda-e379d7dce2eb" (UID: "3b991d83-6dd3-4b34-bdda-e379d7dce2eb"). InnerVolumeSpecName "kube-api-access-698dq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 13 01:24:26.330490 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.330455 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-extensions-socket-volume\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:24:26.330490 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.330484 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-698dq\" (UniqueName: \"kubernetes.io/projected/3b991d83-6dd3-4b34-bdda-e379d7dce2eb-kube-api-access-698dq\") on node \"ip-10-0-137-107.ec2.internal\" DevicePath \"\"" Jun 13 01:24:26.841191 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.841156 2552 generic.go:356] "Generic (PLEG): container finished" podID="3b991d83-6dd3-4b34-bdda-e379d7dce2eb" containerID="aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378" exitCode=0 Jun 13 01:24:26.841368 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.841197 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" Jun 13 01:24:26.841368 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.841240 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" event={"ID":"3b991d83-6dd3-4b34-bdda-e379d7dce2eb","Type":"ContainerDied","Data":"aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378"} Jun 13 01:24:26.841368 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.841291 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9" event={"ID":"3b991d83-6dd3-4b34-bdda-e379d7dce2eb","Type":"ContainerDied","Data":"3c00e117da7761dd424d4e1fdfecc994e1438fc455a8e67ac34e036f1decb8d9"} Jun 13 01:24:26.841368 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.841315 2552 scope.go:117] "RemoveContainer" containerID="aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378" Jun 13 01:24:26.851651 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.851635 2552 scope.go:117] "RemoveContainer" containerID="aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378" Jun 13 01:24:26.851909 ip-10-0-137-107 kubenswrapper[2552]: E0613 01:24:26.851887 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378\": container with ID starting with aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378 not found: ID does not exist" containerID="aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378" Jun 13 01:24:26.852021 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.851915 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378"} err="failed to get container status \"aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378\": rpc error: code = NotFound desc = could not find container \"aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378\": container with ID starting with aee52f03a1ca5071a6a7bd578afa4f4a4b160c857e0e3c9b7912685cc33c5378 not found: ID does not exist" Jun 13 01:24:26.864208 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.864186 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9"] Jun 13 01:24:26.867691 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:26.867668 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dvkx9"] Jun 13 01:24:28.855199 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:24:28.855167 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b991d83-6dd3-4b34-bdda-e379d7dce2eb" path="/var/lib/kubelet/pods/3b991d83-6dd3-4b34-bdda-e379d7dce2eb/volumes" Jun 13 01:25:27.009641 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.009605 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j"] Jun 13 01:25:27.010080 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.009994 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b991d83-6dd3-4b34-bdda-e379d7dce2eb" containerName="manager" Jun 13 01:25:27.010080 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.010005 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b991d83-6dd3-4b34-bdda-e379d7dce2eb" containerName="manager" Jun 13 01:25:27.010164 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.010087 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b991d83-6dd3-4b34-bdda-e379d7dce2eb" containerName="manager" Jun 13 01:25:27.013035 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.013019 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:25:27.015404 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.015383 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-frk59\"" Jun 13 01:25:27.025177 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.025150 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csnnn\" (UniqueName: \"kubernetes.io/projected/4529618d-5e1b-426e-b4b1-a5b10d2f325d-kube-api-access-csnnn\") pod \"kuadrant-operator-controller-manager-86878c769f-n695j\" (UID: \"4529618d-5e1b-426e-b4b1-a5b10d2f325d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:25:27.025177 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.025169 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j"] Jun 13 01:25:27.025326 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.025188 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4529618d-5e1b-426e-b4b1-a5b10d2f325d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n695j\" (UID: \"4529618d-5e1b-426e-b4b1-a5b10d2f325d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:25:27.126167 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.126142 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csnnn\" (UniqueName: \"kubernetes.io/projected/4529618d-5e1b-426e-b4b1-a5b10d2f325d-kube-api-access-csnnn\") pod \"kuadrant-operator-controller-manager-86878c769f-n695j\" (UID: \"4529618d-5e1b-426e-b4b1-a5b10d2f325d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:25:27.126280 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.126173 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4529618d-5e1b-426e-b4b1-a5b10d2f325d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n695j\" (UID: \"4529618d-5e1b-426e-b4b1-a5b10d2f325d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:25:27.126471 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.126456 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4529618d-5e1b-426e-b4b1-a5b10d2f325d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n695j\" (UID: \"4529618d-5e1b-426e-b4b1-a5b10d2f325d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:25:27.134335 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.134305 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csnnn\" (UniqueName: \"kubernetes.io/projected/4529618d-5e1b-426e-b4b1-a5b10d2f325d-kube-api-access-csnnn\") pod \"kuadrant-operator-controller-manager-86878c769f-n695j\" (UID: \"4529618d-5e1b-426e-b4b1-a5b10d2f325d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:25:27.323120 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.323095 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:25:27.463018 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:27.462993 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j"] Jun 13 01:25:27.464828 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:25:27.464752 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4529618d_5e1b_426e_b4b1_a5b10d2f325d.slice/crio-07e44dee62b3392a8c8329c34bb252273b11005d061c17d581e443a76131af91 WatchSource:0}: Error finding container 07e44dee62b3392a8c8329c34bb252273b11005d061c17d581e443a76131af91: Status 404 returned error can't find the container with id 07e44dee62b3392a8c8329c34bb252273b11005d061c17d581e443a76131af91 Jun 13 01:25:28.146599 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:28.146566 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" event={"ID":"4529618d-5e1b-426e-b4b1-a5b10d2f325d","Type":"ContainerStarted","Data":"b0a92e7c79ef70c102f4475c71b1a17aaf064f6acc81d714959d797904a7c8d1"} Jun 13 01:25:28.147074 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:28.146619 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" event={"ID":"4529618d-5e1b-426e-b4b1-a5b10d2f325d","Type":"ContainerStarted","Data":"07e44dee62b3392a8c8329c34bb252273b11005d061c17d581e443a76131af91"} Jun 13 01:25:28.147074 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:28.146687 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:25:28.166677 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:28.166627 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" podStartSLOduration=2.166608519 podStartE2EDuration="2.166608519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-13 01:25:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:25:28.164517979 +0000 UTC m=+1767.738446895" watchObservedRunningTime="2026-06-13 01:25:28.166608519 +0000 UTC m=+1767.740537407" Jun 13 01:25:39.154256 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:25:39.154228 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n695j" Jun 13 01:26:00.971539 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:26:00.971437 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:26:00.980392 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:26:00.976838 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:31:01.007111 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:31:01.006985 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:31:01.013107 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:31:01.013083 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:35:14.891663 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:14.891631 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-86dc5dcf99-z59ht_2ec3aeec-0939-41bc-86f7-05e36d05951f/nginx/0.log" Jun 13 01:35:16.037122 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:16.037094 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78dd6f457b-g7nx5_e26902e2-7b84-4a5d-a93a-de08abe4de05/copy-manifests/0.log" Jun 13 01:35:16.043305 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:16.043286 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78dd6f457b-g7nx5_e26902e2-7b84-4a5d-a93a-de08abe4de05/manager/0.log" Jun 13 01:35:17.634401 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:17.634369 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fngbx_59571565-bb5a-475e-b92c-45b493d1dc54/manager/0.log" Jun 13 01:35:17.749380 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:17.749359 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8wmd8_c8047eb1-2cd5-41b6-acc3-c7c753a2b615/manager/0.log" Jun 13 01:35:17.975671 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:17.975614 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8mztn_561f4c8b-a710-4fe7-87e0-3b1c7270c6bc/registry-server/0.log" Jun 13 01:35:18.092912 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:18.092888 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-n695j_4529618d-5e1b-426e-b4b1-a5b10d2f325d/manager/0.log" Jun 13 01:35:18.673144 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:18.673121 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc9tgn8_c986a91d-2ddd-4f79-91d4-9e86a7a475f8/istio-proxy/0.log" Jun 13 01:35:19.117477 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:19.117454 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-pdnf7_f4876ece-d3c1-45a5-9c59-300d782c7bdc/istio-proxy/1.log" Jun 13 01:35:19.227290 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:19.227263 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8c5m4_20f5b379-eeb8-4c02-9e97-70d8c7d29792/payload-pre-processing/0.log" Jun 13 01:35:19.337689 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:19.337665 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-cvl8j_90eb8af4-d86b-42d2-836d-85b8c4d69029/payload-processing/0.log" Jun 13 01:35:19.446274 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:19.446215 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-788bb8b696-bhgjk_c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036/router/0.log" Jun 13 01:35:19.780851 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:19.780793 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn_2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e/storage-initializer/0.log" Jun 13 01:35:19.787548 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:19.787529 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-bw2sn_2c27fdd0-55b4-4cd7-ac5b-7bf9a85cb92e/main/0.log" Jun 13 01:35:20.012244 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:20.012219 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs_7c9d6f93-c84f-4bff-8050-4ea64cdd328c/storage-initializer/0.log" Jun 13 01:35:20.018929 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:20.018912 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-4bfxs_7c9d6f93-c84f-4bff-8050-4ea64cdd328c/main/0.log" Jun 13 01:35:20.128580 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:20.128562 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln_f48efb87-ef25-4c57-b84e-b63b2293c21f/storage-initializer/0.log" Jun 13 01:35:20.134811 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:20.134794 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cltnln_f48efb87-ef25-4c57-b84e-b63b2293c21f/main/0.log" Jun 13 01:35:20.240761 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:20.240743 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj_fa08828e-98fc-4431-9c34-d35e52670859/storage-initializer/0.log" Jun 13 01:35:20.247660 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:20.247644 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-tbrpj_fa08828e-98fc-4431-9c34-d35e52670859/main/0.log" Jun 13 01:35:27.391124 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:27.391093 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5sblb_ae75009f-6dcc-49b0-8d6a-56b6e2813f00/global-pull-secret-syncer/0.log" Jun 13 01:35:27.574203 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:27.574173 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wntlh_a671b759-54ea-4e80-8f31-e3e392510140/konnectivity-agent/0.log" Jun 13 01:35:27.640080 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:27.640055 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-107.ec2.internal_86dcde8117c8f08bff9dfbde122f9160/haproxy/0.log" Jun 13 01:35:32.892526 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:32.892500 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fngbx_59571565-bb5a-475e-b92c-45b493d1dc54/manager/0.log" Jun 13 01:35:32.921750 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:32.921726 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8wmd8_c8047eb1-2cd5-41b6-acc3-c7c753a2b615/manager/0.log" Jun 13 01:35:32.984640 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:32.984617 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8mztn_561f4c8b-a710-4fe7-87e0-3b1c7270c6bc/registry-server/0.log" Jun 13 01:35:33.045776 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:33.045757 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-n695j_4529618d-5e1b-426e-b4b1-a5b10d2f325d/manager/0.log" Jun 13 01:35:34.772035 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:34.772008 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-f9d7df769-2cjck_210bbc79-fa75-47da-9a3e-69b702fa8cff/cluster-monitoring-operator/0.log" Jun 13 01:35:35.036938 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.036917 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-97l2g_c91d0129-4b62-4ae5-a5d2-ba2d23846ec1/node-exporter/0.log" Jun 13 01:35:35.060680 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.060659 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-97l2g_c91d0129-4b62-4ae5-a5d2-ba2d23846ec1/kube-rbac-proxy/0.log" Jun 13 01:35:35.084280 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.084263 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-97l2g_c91d0129-4b62-4ae5-a5d2-ba2d23846ec1/init-textfile/0.log" Jun 13 01:35:35.315656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.315635 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71cad2aa-4868-4997-8a27-caa57306c0e4/prometheus/0.log" Jun 13 01:35:35.340099 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.340083 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71cad2aa-4868-4997-8a27-caa57306c0e4/config-reloader/0.log" Jun 13 01:35:35.367656 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.367641 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71cad2aa-4868-4997-8a27-caa57306c0e4/thanos-sidecar/0.log" Jun 13 01:35:35.405040 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.405023 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71cad2aa-4868-4997-8a27-caa57306c0e4/kube-rbac-proxy-web/0.log" Jun 13 01:35:35.432071 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.432056 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71cad2aa-4868-4997-8a27-caa57306c0e4/kube-rbac-proxy/0.log" Jun 13 01:35:35.454103 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.454083 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71cad2aa-4868-4997-8a27-caa57306c0e4/kube-rbac-proxy-thanos/0.log" Jun 13 01:35:35.473958 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.473944 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71cad2aa-4868-4997-8a27-caa57306c0e4/init-config-reloader/0.log" Jun 13 01:35:35.500803 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.500772 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-864fcfc6f9-5d4z8_7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d/prometheus-operator/0.log" Jun 13 01:35:35.520982 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:35.520963 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-864fcfc6f9-5d4z8_7a5a33f9-3cc9-4d5b-ab3f-aa100f256a4d/kube-rbac-proxy/0.log" Jun 13 01:35:37.175254 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:37.175224 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-867b76cb74-tq4xr_7140ef67-07de-4771-b594-0a3c63f7254c/networking-console-plugin/0.log" Jun 13 01:35:38.393010 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:38.392981 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-694c955b9-w2g79_3e53de96-99fe-4974-bddf-4f033e866518/download-server/0.log" Jun 13 01:35:39.042452 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.042424 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-bd4ccd799-qjn9z_1622fbd1-cbee-4f68-a7ae-8a7b410dd1e6/volume-data-source-validator/0.log" Jun 13 01:35:39.437175 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.437145 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz"] Jun 13 01:35:39.440648 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.440614 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.443029 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.443004 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2hv7p\"/\"openshift-service-ca.crt\"" Jun 13 01:35:39.443164 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.443052 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2hv7p\"/\"kube-root-ca.crt\"" Jun 13 01:35:39.444014 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.443999 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2hv7p\"/\"default-dockercfg-np42k\"" Jun 13 01:35:39.449434 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.449414 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz"] Jun 13 01:35:39.521612 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.521585 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-sys\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.521734 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.521637 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-proc\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.521734 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.521702 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24r85\" (UniqueName: \"kubernetes.io/projected/93dfdbca-b96a-40e3-ab72-e50bb8f752da-kube-api-access-24r85\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.521734 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.521729 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-podres\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.521872 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.521764 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-lib-modules\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.622965 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.622932 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-sys\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.623108 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.622973 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-proc\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.623108 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.623026 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24r85\" (UniqueName: \"kubernetes.io/projected/93dfdbca-b96a-40e3-ab72-e50bb8f752da-kube-api-access-24r85\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.623108 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.623051 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-podres\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.623108 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.623053 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-sys\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.623258 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.623114 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-lib-modules\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.623258 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.623133 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-proc\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.623258 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.623186 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-podres\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.623258 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.623237 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93dfdbca-b96a-40e3-ab72-e50bb8f752da-lib-modules\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.630866 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.630840 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24r85\" (UniqueName: \"kubernetes.io/projected/93dfdbca-b96a-40e3-ab72-e50bb8f752da-kube-api-access-24r85\") pod \"perf-node-gather-daemonset-6kcfz\" (UID: \"93dfdbca-b96a-40e3-ab72-e50bb8f752da\") " pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.751479 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.751417 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:39.878844 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:39.878762 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz"] Jun 13 01:35:39.880799 ip-10-0-137-107 kubenswrapper[2552]: W0613 01:35:39.880760 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod93dfdbca_b96a_40e3_ab72_e50bb8f752da.slice/crio-829ec783a9497cb0b362b96f2345174525b11580c9aff9e818d5c85c61cffaf8 WatchSource:0}: Error finding container 829ec783a9497cb0b362b96f2345174525b11580c9aff9e818d5c85c61cffaf8: Status 404 returned error can't find the container with id 829ec783a9497cb0b362b96f2345174525b11580c9aff9e818d5c85c61cffaf8 Jun 13 01:35:40.280829 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:40.280797 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" event={"ID":"93dfdbca-b96a-40e3-ab72-e50bb8f752da","Type":"ContainerStarted","Data":"f0558493390db51d040d64077a29df39ef022bb05f8cc961a584128fcab3f355"} Jun 13 01:35:40.280984 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:40.280842 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" event={"ID":"93dfdbca-b96a-40e3-ab72-e50bb8f752da","Type":"ContainerStarted","Data":"829ec783a9497cb0b362b96f2345174525b11580c9aff9e818d5c85c61cffaf8"} Jun 13 01:35:40.280984 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:40.280909 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:40.297951 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:40.297895 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" podStartSLOduration=1.297882427 podStartE2EDuration="1.297882427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-13 01:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-13 01:35:40.296104247 +0000 UTC m=+2379.870033134" watchObservedRunningTime="2026-06-13 01:35:40.297882427 +0000 UTC m=+2379.871811314" Jun 13 01:35:40.309908 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:40.309893 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f5lsk_d35bfb84-8978-47a6-8c05-034fb69773dd/dns/0.log" Jun 13 01:35:40.335630 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:40.335610 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f5lsk_d35bfb84-8978-47a6-8c05-034fb69773dd/kube-rbac-proxy/0.log" Jun 13 01:35:40.491466 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:40.491444 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xf2jd_87376b5e-f663-4b36-affd-55da108f6af7/dns-node-resolver/0.log" Jun 13 01:35:41.068864 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:41.068836 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dvssx_08c13986-e45f-4f66-a34e-0401741bdfcf/node-ca/0.log" Jun 13 01:35:42.019999 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:42.019967 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc9tgn8_c986a91d-2ddd-4f79-91d4-9e86a7a475f8/istio-proxy/0.log" Jun 13 01:35:42.154206 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:42.154179 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-pdnf7_f4876ece-d3c1-45a5-9c59-300d782c7bdc/istio-proxy/0.log" Jun 13 01:35:42.183790 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:42.183765 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-pdnf7_f4876ece-d3c1-45a5-9c59-300d782c7bdc/istio-proxy/1.log" Jun 13 01:35:42.212111 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:42.212090 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8c5m4_20f5b379-eeb8-4c02-9e97-70d8c7d29792/payload-pre-processing/0.log" Jun 13 01:35:42.273180 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:42.273122 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-cvl8j_90eb8af4-d86b-42d2-836d-85b8c4d69029/payload-processing/0.log" Jun 13 01:35:42.297032 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:42.297012 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-788bb8b696-bhgjk_c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036/router/0.log" Jun 13 01:35:42.929534 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:42.929508 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w4kkf_fa4f439a-83ef-4ba0-a9c6-ae5d1655d033/serve-healthcheck-canary/0.log" Jun 13 01:35:43.417329 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:43.417307 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5bbd86d6bd-7f6mh_3c53466a-47d5-4367-ba1f-de324cb84360/insights-operator/0.log" Jun 13 01:35:43.417811 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:43.417793 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5bbd86d6bd-7f6mh_3c53466a-47d5-4367-ba1f-de324cb84360/insights-operator/1.log" Jun 13 01:35:43.504279 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:43.504258 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kkm5c_fb25f6cb-295d-4c25-8614-f3e2a02d4538/kube-rbac-proxy/0.log" Jun 13 01:35:43.523643 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:43.523621 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kkm5c_fb25f6cb-295d-4c25-8614-f3e2a02d4538/exporter/0.log" Jun 13 01:35:43.543698 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:43.543679 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kkm5c_fb25f6cb-295d-4c25-8614-f3e2a02d4538/extractor/0.log" Jun 13 01:35:45.773341 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:45.773315 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-86dc5dcf99-z59ht_2ec3aeec-0939-41bc-86f7-05e36d05951f/nginx/0.log" Jun 13 01:35:46.228342 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:46.228297 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78dd6f457b-g7nx5_e26902e2-7b84-4a5d-a93a-de08abe4de05/manager/0.log" Jun 13 01:35:46.251291 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:46.251272 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78dd6f457b-g7nx5_e26902e2-7b84-4a5d-a93a-de08abe4de05/copy-manifests/0.log" Jun 13 01:35:46.295325 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:46.295308 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2hv7p/perf-node-gather-daemonset-6kcfz" Jun 13 01:35:48.063543 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:48.063520 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6dc4584f75-nbvpp_94606d7c-075f-4b0a-abdf-5adb1ccb6823/manager/0.log" Jun 13 01:35:53.036299 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:53.036266 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc9tgn8_c986a91d-2ddd-4f79-91d4-9e86a7a475f8/istio-proxy/0.log" Jun 13 01:35:53.138068 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:53.138035 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-pdnf7_f4876ece-d3c1-45a5-9c59-300d782c7bdc/istio-proxy/0.log" Jun 13 01:35:53.169882 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:53.169858 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-pdnf7_f4876ece-d3c1-45a5-9c59-300d782c7bdc/istio-proxy/1.log" Jun 13 01:35:53.187681 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:53.187656 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8c5m4_20f5b379-eeb8-4c02-9e97-70d8c7d29792/payload-pre-processing/0.log" Jun 13 01:35:53.237859 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:53.237836 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-cvl8j_90eb8af4-d86b-42d2-836d-85b8c4d69029/payload-processing/0.log" Jun 13 01:35:53.255698 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:53.255677 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-788bb8b696-bhgjk_c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036/router/0.log" Jun 13 01:35:53.880825 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:53.880792 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7c578b78fc-lchjn_e85773c6-ab14-45ea-8df1-20975bf51131/kube-storage-version-migrator-operator/1.log" Jun 13 01:35:53.881628 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:53.881609 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7c578b78fc-lchjn_e85773c6-ab14-45ea-8df1-20975bf51131/kube-storage-version-migrator-operator/0.log" Jun 13 01:35:54.379263 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:54.379231 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc9tgn8_c986a91d-2ddd-4f79-91d4-9e86a7a475f8/istio-proxy/0.log" Jun 13 01:35:54.490685 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:54.490659 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-pdnf7_f4876ece-d3c1-45a5-9c59-300d782c7bdc/istio-proxy/0.log" Jun 13 01:35:54.520065 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:54.520041 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-pdnf7_f4876ece-d3c1-45a5-9c59-300d782c7bdc/istio-proxy/1.log" Jun 13 01:35:54.540380 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:54.540358 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8c5m4_20f5b379-eeb8-4c02-9e97-70d8c7d29792/payload-pre-processing/0.log" Jun 13 01:35:54.593271 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:54.593245 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-cvl8j_90eb8af4-d86b-42d2-836d-85b8c4d69029/payload-processing/0.log" Jun 13 01:35:54.607033 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:54.607009 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-788bb8b696-bhgjk_c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036/router/0.log" Jun 13 01:35:55.364179 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.364148 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tqx2r_638de231-57e4-4732-bc73-794b751a3249/kube-multus-additional-cni-plugins/0.log" Jun 13 01:35:55.424037 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.424007 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tqx2r_638de231-57e4-4732-bc73-794b751a3249/egress-router-binary-copy/0.log" Jun 13 01:35:55.475717 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.475697 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tqx2r_638de231-57e4-4732-bc73-794b751a3249/cni-plugins/0.log" Jun 13 01:35:55.507700 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.507643 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tqx2r_638de231-57e4-4732-bc73-794b751a3249/bond-cni-plugin/0.log" Jun 13 01:35:55.527370 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.527350 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tqx2r_638de231-57e4-4732-bc73-794b751a3249/routeoverride-cni/0.log" Jun 13 01:35:55.547332 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.547317 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tqx2r_638de231-57e4-4732-bc73-794b751a3249/whereabouts-cni-bincopy/0.log" Jun 13 01:35:55.570830 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.570813 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tqx2r_638de231-57e4-4732-bc73-794b751a3249/whereabouts-cni/0.log" Jun 13 01:35:55.600435 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.600417 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fmrlv_d69a9eb7-5078-49c0-a3c2-82452c082d79/kube-multus/0.log" Jun 13 01:35:55.704254 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.704229 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ln4ft_7ae3796c-d916-4f7d-ac66-24899ca4e0f0/network-metrics-daemon/0.log" Jun 13 01:35:55.724975 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:55.724956 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ln4ft_7ae3796c-d916-4f7d-ac66-24899ca4e0f0/kube-rbac-proxy/0.log" Jun 13 01:35:56.167026 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.167001 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bc9tgn8_c986a91d-2ddd-4f79-91d4-9e86a7a475f8/istio-proxy/0.log" Jun 13 01:35:56.272894 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.272865 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-pdnf7_f4876ece-d3c1-45a5-9c59-300d782c7bdc/istio-proxy/0.log" Jun 13 01:35:56.303112 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.303086 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-pdnf7_f4876ece-d3c1-45a5-9c59-300d782c7bdc/istio-proxy/1.log" Jun 13 01:35:56.323332 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.323310 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-8c5m4_20f5b379-eeb8-4c02-9e97-70d8c7d29792/payload-pre-processing/0.log" Jun 13 01:35:56.371253 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.371225 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-cvl8j_90eb8af4-d86b-42d2-836d-85b8c4d69029/payload-processing/0.log" Jun 13 01:35:56.385482 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.385464 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-788bb8b696-bhgjk_c9bcb0a1-3cc5-40e2-8d3c-30b54f1f3036/router/0.log" Jun 13 01:35:56.713633 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.713607 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-controller/0.log" Jun 13 01:35:56.730972 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.730946 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/1.log" Jun 13 01:35:56.731111 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.730970 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovn-acl-logging/0.log" Jun 13 01:35:56.751625 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.751599 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/kube-rbac-proxy-node/0.log" Jun 13 01:35:56.774263 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.774242 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/kube-rbac-proxy-ovn-metrics/0.log" Jun 13 01:35:56.790712 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.790692 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/northd/0.log" Jun 13 01:35:56.812806 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.812773 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/nbdb/0.log" Jun 13 01:35:56.834235 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.834216 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/sbdb/0.log" Jun 13 01:35:56.879246 ip-10-0-137-107 kubenswrapper[2552]: I0613 01:35:56.879227 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2998n_85c21015-b7b2-4d3a-876d-e551b28b516f/ovnkube-controller/0.log"