Sep 19 04:15:32.216481 ip-10-0-132-168 systemd[1]: Starting Kubernetes Kubelet... Sep 19 04:15:35.650817 ip-10-0-132-168 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 04:15:35.650817 ip-10-0-132-168 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 19 04:15:35.650817 ip-10-0-132-168 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 04:15:35.650817 ip-10-0-132-168 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 19 04:15:35.650817 ip-10-0-132-168 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 04:15:35.651628 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.650902 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 19 04:15:35.654966 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654944 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 04:15:35.654966 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654962 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 04:15:35.654966 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654967 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 04:15:35.654966 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654972 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654977 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654981 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654984 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654988 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654993 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.654996 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655000 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655004 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655008 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655011 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655015 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655018 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655022 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655025 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655039 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655043 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655046 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655050 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655054 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 04:15:35.655240 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655058 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655062 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655065 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655069 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655072 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655076 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655080 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655084 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655088 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655092 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655096 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655100 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655104 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655109 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655114 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655118 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655123 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655127 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655132 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 04:15:35.656045 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655135 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655140 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655144 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655164 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655170 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655174 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655179 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655183 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655189 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655193 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655197 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655203 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655207 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655211 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655215 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655219 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655223 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655228 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655232 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655238 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 04:15:35.656853 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655245 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655249 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655253 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655257 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655261 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655265 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655270 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655276 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655280 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655285 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655289 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655293 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655297 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655301 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655306 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655310 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655314 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655318 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655325 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655329 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 04:15:35.657699 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655333 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655338 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655343 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655346 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655350 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655355 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655359 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655366 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655371 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655377 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655381 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655387 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655393 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655398 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655403 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655408 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.655412 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656573 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656584 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 04:15:35.658568 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656590 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656594 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656599 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656605 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656609 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656613 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656617 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656621 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656626 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656630 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656634 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656638 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656642 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656646 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656650 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656655 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656660 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656664 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656670 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 04:15:35.659389 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656677 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656681 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656685 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656689 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656693 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656700 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656705 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656709 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656714 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656718 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656723 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656728 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656732 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656738 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656743 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656748 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656752 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656757 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656761 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 04:15:35.660044 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656765 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656769 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656774 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656778 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656781 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656786 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656790 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656794 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656798 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656802 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656807 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656811 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656815 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656821 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656826 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656831 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656835 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656839 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656843 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656846 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 04:15:35.660602 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656850 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656854 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656858 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656861 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656865 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656869 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656874 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656879 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656886 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656894 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656898 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656903 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656907 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656912 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656916 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656920 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656924 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656928 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656933 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656936 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 04:15:35.661125 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656940 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656944 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656948 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656953 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656957 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656962 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656966 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656970 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656974 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656979 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656983 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656987 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656991 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656995 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.656999 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657003 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657007 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657011 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657016 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657696 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 04:15:35.661673 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657706 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657711 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657715 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657722 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657728 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657733 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657738 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657742 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657746 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657751 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657757 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657762 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657766 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657770 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657774 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657781 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657786 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657790 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657795 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 04:15:35.662162 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657799 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657803 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657807 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657811 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657815 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657818 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657823 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657828 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657832 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657836 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657840 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657848 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657852 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657856 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657861 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657865 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657869 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657873 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657877 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657881 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 04:15:35.662865 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657885 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657891 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657895 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657899 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657903 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657909 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657915 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657920 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657924 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657929 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657933 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657937 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657942 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657946 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657950 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657954 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657959 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657963 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657967 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657972 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 04:15:35.663384 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657977 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657981 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657985 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657991 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657995 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.657999 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658004 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658008 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658012 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658018 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658022 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658025 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658030 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658034 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658038 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658042 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658046 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658050 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658056 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 04:15:35.663912 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658062 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658066 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658071 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658075 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658082 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658086 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658090 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658094 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658098 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658102 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658106 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658110 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658114 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658118 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658123 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658127 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658132 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658136 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658141 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.658145 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 04:15:35.664454 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658263 2577 flags.go:64] FLAG: --address="0.0.0.0" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658318 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658328 2577 flags.go:64] FLAG: --anonymous-auth="true" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658334 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658341 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658346 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658353 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658360 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658365 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658369 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658375 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658381 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658386 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658391 2577 flags.go:64] FLAG: --cgroup-root="" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658395 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658400 2577 flags.go:64] FLAG: --client-ca-file="" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658408 2577 flags.go:64] FLAG: --cloud-provider="external" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658413 2577 flags.go:64] FLAG: --cluster-dns="[]" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658418 2577 flags.go:64] FLAG: --cluster-domain="" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658423 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658428 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658434 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658439 2577 flags.go:64] FLAG: --container-log-max-files="5" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658445 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 19 04:15:35.664977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658449 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658454 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658459 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658465 2577 flags.go:64] FLAG: --contention-profiling="false" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658471 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658476 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658481 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658486 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658492 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658497 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658501 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658507 2577 flags.go:64] FLAG: --enable-load-reader="false" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658511 2577 flags.go:64] FLAG: --enable-server="true" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658516 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658522 2577 flags.go:64] FLAG: --event-burst="100" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658527 2577 flags.go:64] FLAG: --event-qps="50" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658532 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658537 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658542 2577 flags.go:64] FLAG: --eviction-hard="" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658548 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658553 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658558 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658562 2577 flags.go:64] FLAG: --eviction-soft="" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658567 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658573 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 19 04:15:35.665609 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658578 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658582 2577 flags.go:64] FLAG: --experimental-mounter-path="" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658587 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658592 2577 flags.go:64] FLAG: --fail-swap-on="true" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658597 2577 flags.go:64] FLAG: --feature-gates="" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658603 2577 flags.go:64] FLAG: --file-check-frequency="20s" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658608 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658613 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658618 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658623 2577 flags.go:64] FLAG: --healthz-port="10248" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658628 2577 flags.go:64] FLAG: --help="false" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658634 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-132-168.ec2.internal" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658639 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658644 2577 flags.go:64] FLAG: --http-check-frequency="20s" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658650 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658655 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658661 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658665 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658670 2577 flags.go:64] FLAG: --image-service-endpoint="" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658674 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658679 2577 flags.go:64] FLAG: --kube-api-burst="100" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658683 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658688 2577 flags.go:64] FLAG: --kube-api-qps="50" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658693 2577 flags.go:64] FLAG: --kube-reserved="" Sep 19 04:15:35.666247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658698 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658703 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658708 2577 flags.go:64] FLAG: --kubelet-cgroups="" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658712 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658716 2577 flags.go:64] FLAG: --lock-file="" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658721 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658726 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658731 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658739 2577 flags.go:64] FLAG: --log-json-split-stream="false" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658745 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658749 2577 flags.go:64] FLAG: --log-text-split-stream="false" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658754 2577 flags.go:64] FLAG: --logging-format="text" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658759 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658764 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658768 2577 flags.go:64] FLAG: --manifest-url="" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658773 2577 flags.go:64] FLAG: --manifest-url-header="" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658780 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658785 2577 flags.go:64] FLAG: --max-open-files="1000000" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658791 2577 flags.go:64] FLAG: --max-pods="110" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658797 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658802 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658806 2577 flags.go:64] FLAG: --memory-manager-policy="None" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658811 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658816 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658821 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 19 04:15:35.666876 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658826 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658836 2577 flags.go:64] FLAG: --node-status-max-images="50" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658841 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658845 2577 flags.go:64] FLAG: --oom-score-adj="-999" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658850 2577 flags.go:64] FLAG: --pod-cidr="" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658855 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658864 2577 flags.go:64] FLAG: --pod-manifest-path="" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658868 2577 flags.go:64] FLAG: --pod-max-pids="-1" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658873 2577 flags.go:64] FLAG: --pods-per-core="0" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658878 2577 flags.go:64] FLAG: --port="10250" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658883 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658887 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ea7d4ab8aec9776e" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658892 2577 flags.go:64] FLAG: --qos-reserved="" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658897 2577 flags.go:64] FLAG: --read-only-port="10255" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658902 2577 flags.go:64] FLAG: --register-node="true" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658907 2577 flags.go:64] FLAG: --register-with-taints="" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658913 2577 flags.go:64] FLAG: --registry-burst="10" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658918 2577 flags.go:64] FLAG: --registry-qps="5" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658923 2577 flags.go:64] FLAG: --reserved-cpus="" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658927 2577 flags.go:64] FLAG: --reserved-memory="" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658934 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658939 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658944 2577 flags.go:64] FLAG: --rotate-certificates="false" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658948 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658953 2577 flags.go:64] FLAG: --runonce="false" Sep 19 04:15:35.667813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658958 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658963 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658969 2577 flags.go:64] FLAG: --seccomp-default="false" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658974 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658978 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658983 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658988 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658993 2577 flags.go:64] FLAG: --storage-driver-password="root" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.658998 2577 flags.go:64] FLAG: --storage-driver-secure="false" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659002 2577 flags.go:64] FLAG: --storage-driver-table="stats" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659007 2577 flags.go:64] FLAG: --storage-driver-user="root" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659012 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659017 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659022 2577 flags.go:64] FLAG: --system-cgroups="" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659027 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659036 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659040 2577 flags.go:64] FLAG: --tls-cert-file="" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659044 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659049 2577 flags.go:64] FLAG: --tls-min-version="" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659054 2577 flags.go:64] FLAG: --tls-private-key-file="" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659059 2577 flags.go:64] FLAG: --topology-manager-policy="none" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659064 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659079 2577 flags.go:64] FLAG: --topology-manager-scope="container" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659085 2577 flags.go:64] FLAG: --v="2" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659091 2577 flags.go:64] FLAG: --version="false" Sep 19 04:15:35.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659098 2577 flags.go:64] FLAG: --vmodule="" Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659104 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.659110 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659308 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659316 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659321 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659326 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659334 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659340 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659346 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659353 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659358 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659363 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659368 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659372 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659378 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659383 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659388 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659392 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 04:15:35.669694 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659396 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659400 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659404 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659408 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659412 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659416 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659420 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659425 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659430 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659434 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659439 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659445 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659450 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659454 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659460 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659464 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659469 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659473 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659477 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659481 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 04:15:35.670192 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659485 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659489 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659496 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659500 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659504 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659508 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659513 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659517 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659521 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659525 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659529 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659533 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659538 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659541 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659546 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659550 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659554 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659558 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659562 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 04:15:35.670677 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659566 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659570 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659576 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659581 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659585 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659588 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659592 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659597 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659601 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659606 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659610 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659613 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659618 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659622 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659626 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659632 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659636 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659640 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659644 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659648 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 04:15:35.671132 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659653 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659658 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659662 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659666 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659670 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659674 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659679 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659683 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659687 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659692 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659696 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659700 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659704 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659708 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659714 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659718 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659722 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659726 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659730 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659734 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 04:15:35.671648 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659738 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659743 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659747 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.659751 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.660445 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.671026 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.671043 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.671066 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.671072 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671172 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671179 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671182 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671185 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671188 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 04:15:35.672142 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671191 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671193 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671196 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671199 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671202 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671204 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671207 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671209 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671212 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671215 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671217 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671220 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671223 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671226 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671228 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671231 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671234 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671236 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671238 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671241 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 04:15:35.672513 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671243 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671246 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671248 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671251 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671255 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671257 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671260 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671263 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671266 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671269 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671271 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671274 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671276 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671279 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671281 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671284 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671287 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671289 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671292 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671294 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 04:15:35.673004 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671297 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671299 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671302 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671305 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671308 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671311 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671313 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671316 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671318 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671320 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671323 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671327 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671331 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671334 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671337 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671341 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671345 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671348 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671351 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 04:15:35.673615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671354 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671357 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671360 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671363 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671365 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671368 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671370 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671373 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671375 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671378 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671380 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671383 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671385 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671387 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671390 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671393 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671396 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671399 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671401 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671403 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671406 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 04:15:35.674072 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671409 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671411 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671414 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671416 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671419 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671421 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671425 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671428 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671431 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671434 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671437 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671440 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671443 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671446 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 04:15:35.674615 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.671451 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671573 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671579 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671581 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671584 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671587 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671590 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671593 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671595 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671598 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671601 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671603 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671606 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671611 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671613 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671616 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671618 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671621 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671623 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671626 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 04:15:35.674971 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671628 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671631 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671634 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671637 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671639 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671642 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671644 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671648 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671652 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671657 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671660 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671663 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671665 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671669 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671673 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671676 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671678 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671681 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671684 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 04:15:35.675473 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671686 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671689 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671691 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671694 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671696 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671699 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671701 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671704 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671707 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671709 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671712 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671714 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671717 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671719 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671721 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671724 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671727 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671729 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671731 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 04:15:35.675926 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671734 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671737 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671740 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671742 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671744 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671747 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671750 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671752 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671755 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671757 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671759 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671762 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671764 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671767 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671769 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671771 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671774 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671776 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671778 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671781 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 04:15:35.676404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671783 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671786 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671789 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671791 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671794 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671796 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671798 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671801 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671804 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671807 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671809 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671811 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671816 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671818 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671821 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671823 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671826 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671828 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671830 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671833 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 04:15:35.676886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671835 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 04:15:35.677519 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:35.671838 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 04:15:35.677519 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.671843 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 04:15:35.677519 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.671962 2577 server.go:962] "Client rotation is on, will bootstrap in background" Sep 19 04:15:35.677519 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.674408 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 19 04:15:35.677519 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.675419 2577 server.go:1019] "Starting client certificate rotation" Sep 19 04:15:35.677519 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.675517 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 19 04:15:35.677519 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.675549 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 19 04:15:35.678096 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.678083 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 19 04:15:35.679398 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.679380 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 19 04:15:35.687923 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.687902 2577 log.go:25] "Validated CRI v1 runtime API" Sep 19 04:15:35.689816 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.689797 2577 log.go:25] "Validated CRI v1 image API" Sep 19 04:15:35.690518 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.690506 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 19 04:15:35.692920 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.692900 2577 fs.go:135] Filesystem UUIDs: map[4920e502-150a-4dd6-b888-a4040508fb45:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f6469483-95ec-4793-ba2b-6bc5879bf0b5:/dev/nvme0n1p3] Sep 19 04:15:35.692976 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.692920 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 19 04:15:35.698041 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.697936 2577 manager.go:216] Machine: {Timestamp:2026-09-19 04:15:35.697438838 +0000 UTC m=+2.018536975 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094923 MemoryCapacity:32812146688 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b2c0226f466c6b8e3bcb4f912ce15 SystemUUID:ec2b2c02-26f4-66c6-b8e3-bcb4f912ce15 BootID:ef49852d-0d93-4c2d-95b2-ecc2d3d5914d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406073344 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:dc:34:09:b9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:dc:34:09:b9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:2b:0e:e6:b3:bc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812146688 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 19 04:15:35.698041 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.698037 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 19 04:15:35.698181 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.698117 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 19 04:15:35.699907 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.699878 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 19 04:15:35.700047 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.699910 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 19 04:15:35.701144 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.701126 2577 topology_manager.go:138] "Creating topology manager with none policy" Sep 19 04:15:35.701144 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.701144 2577 container_manager_linux.go:309] "Creating device plugin manager" Sep 19 04:15:35.701274 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.701166 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 19 04:15:35.701274 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.701178 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 19 04:15:35.702577 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.702566 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 19 04:15:35.705549 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.705538 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 19 04:15:35.705661 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.705652 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 19 04:15:35.705695 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.705686 2577 kubelet.go:486] "Attempting to sync node with API server" Sep 19 04:15:35.705722 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.705697 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 19 04:15:35.705722 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.705707 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 19 04:15:35.705722 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.705716 2577 kubelet.go:398] "Adding apiserver pod source" Sep 19 04:15:35.705794 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.705724 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 19 04:15:35.707601 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.707581 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 19 04:15:35.707670 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.707643 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 19 04:15:35.711576 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.711552 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 19 04:15:35.713373 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.713359 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 19 04:15:35.713429 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.713390 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 19 04:15:35.714550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714534 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 19 04:15:35.714550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714553 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714559 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714565 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714573 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714578 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714592 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714597 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714603 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714608 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714616 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714623 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714652 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 19 04:15:35.714653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.714657 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 19 04:15:35.718755 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.718739 2577 server.go:1268] "Started kubelet" Sep 19 04:15:35.718833 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.718811 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 19 04:15:35.718952 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.718835 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 19 04:15:35.719067 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.719000 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 19 04:15:35.719354 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.719336 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 19 04:15:35.719561 ip-10-0-132-168 systemd[1]: Started Kubernetes Kubelet. Sep 19 04:15:35.720265 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.720246 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-168.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 19 04:15:35.721011 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.720998 2577 server.go:310] "Adding debug handlers to kubelet server" Sep 19 04:15:35.723279 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.723241 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 19 04:15:35.724003 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.723986 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 19 04:15:35.725135 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.725108 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 19 04:15:35.725218 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.725108 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 19 04:15:35.729312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.729297 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 19 04:15:35.729397 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.729313 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 19 04:15:35.729456 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.729440 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:35.729932 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.729896 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 19 04:15:35.730083 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.729910 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 19 04:15:35.730083 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.729948 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 19 04:15:35.730083 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.730047 2577 reconstruct.go:97] "Volume reconstruction finished" Sep 19 04:15:35.730083 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.730057 2577 reconciler.go:29] "Reconciler: start to sync state" Sep 19 04:15:35.730959 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.730943 2577 factory.go:55] Registering systemd factory Sep 19 04:15:35.731038 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.730964 2577 factory.go:223] Registration of the systemd container factory successfully Sep 19 04:15:35.731270 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.731246 2577 factory.go:153] Registering CRI-O factory Sep 19 04:15:35.731270 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.731267 2577 factory.go:223] Registration of the crio container factory successfully Sep 19 04:15:35.731372 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.731360 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 19 04:15:35.731431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.731381 2577 factory.go:103] Registering Raw factory Sep 19 04:15:35.731431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.731395 2577 manager.go:1196] Started watching for new ooms in manager Sep 19 04:15:35.732145 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.732121 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 19 04:15:35.732319 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.732294 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 19 04:15:35.732392 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.732328 2577 manager.go:349] Starting recovery of all containers Sep 19 04:15:35.749490 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.749252 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 19 04:15:35.749620 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.749584 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5h6jt" Sep 19 04:15:35.757958 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.757939 2577 manager.go:354] Recovery completed Sep 19 04:15:35.759459 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.759442 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5h6jt" Sep 19 04:15:35.762228 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.762215 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 04:15:35.764092 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.764074 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory" Sep 19 04:15:35.764179 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.764104 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 04:15:35.764179 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.764115 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID" Sep 19 04:15:35.764652 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.764641 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 19 04:15:35.764652 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.764650 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 19 04:15:35.764772 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.764663 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 19 04:15:35.764772 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.764663 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 04:15:35.765469 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.765450 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory" Sep 19 04:15:35.765517 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.765485 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 04:15:35.765517 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.765500 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID" Sep 19 04:15:35.765885 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.765873 2577 policy_none.go:49] "None policy: Start" Sep 19 04:15:35.765942 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.765887 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 19 04:15:35.765942 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.765897 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 19 04:15:35.765942 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.764899 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-168.ec2.internal.18d69dbcb395e662 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-168.ec2.internal,UID:ip-10-0-132-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-168.ec2.internal,},FirstTimestamp:2026-09-19 04:15:35.718708834 +0000 UTC m=+2.039806971,LastTimestamp:2026-09-19 04:15:35.718708834 +0000 UTC m=+2.039806971,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-168.ec2.internal,}" Sep 19 04:15:35.766990 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.766979 2577 policy_none.go:47] "Start" Sep 19 04:15:35.814689 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.814671 2577 manager.go:337] "Starting Device Plugin manager" Sep 19 04:15:35.825272 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.814731 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 19 04:15:35.825272 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.814744 2577 server.go:87] "Starting device plugin registration server" Sep 19 04:15:35.825272 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.815005 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 19 04:15:35.825272 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.815028 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 19 04:15:35.825272 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.815123 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 19 04:15:35.825272 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.815229 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 19 04:15:35.825272 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.815249 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 19 04:15:35.825272 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.815726 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 19 04:15:35.825272 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.815779 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:35.887558 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.887528 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 19 04:15:35.887681 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.887581 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 19 04:15:35.887681 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.887603 2577 kubelet.go:2448] "Starting kubelet main sync loop" Sep 19 04:15:35.887681 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.887637 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 19 04:15:35.890562 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.890543 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 04:15:35.915915 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.915874 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 04:15:35.916891 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.916876 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory" Sep 19 04:15:35.916948 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.916912 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 04:15:35.916948 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.916935 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID" Sep 19 04:15:35.917019 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.916981 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-168.ec2.internal" Sep 19 04:15:35.924601 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.924587 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-168.ec2.internal" Sep 19 04:15:35.924643 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.924609 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-168.ec2.internal\": node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:35.953776 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:35.953751 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:35.988410 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.988368 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"] Sep 19 04:15:35.988531 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.988462 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 04:15:35.989417 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.989403 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory" Sep 19 04:15:35.989492 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.989432 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 04:15:35.989492 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.989442 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID" Sep 19 04:15:35.991202 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.991188 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 04:15:35.991350 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.991332 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" Sep 19 04:15:35.991397 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.991365 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 04:15:35.991977 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.991961 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory" Sep 19 04:15:35.992038 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.991998 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 04:15:35.992038 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.992009 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID" Sep 19 04:15:35.992113 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.991961 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory" Sep 19 04:15:35.992113 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.992072 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 04:15:35.992113 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.992086 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID" Sep 19 04:15:35.993686 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.993669 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" Sep 19 04:15:35.993764 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.993694 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 04:15:35.994758 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.994743 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory" Sep 19 04:15:35.994829 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.994769 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 04:15:35.994829 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:35.994779 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID" Sep 19 04:15:36.020887 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.020870 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-168.ec2.internal\" not found" node="ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.025257 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.025241 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-168.ec2.internal\" not found" node="ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.031417 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.031399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/911c34c29a66f4fca404d6224fced36c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"911c34c29a66f4fca404d6224fced36c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.031465 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.031424 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aba6b42d05a06c053c050c26af669cbb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" (UID: \"aba6b42d05a06c053c050c26af669cbb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.031465 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.031442 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/911c34c29a66f4fca404d6224fced36c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"911c34c29a66f4fca404d6224fced36c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.054514 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.054492 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:36.131830 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.131799 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/911c34c29a66f4fca404d6224fced36c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"911c34c29a66f4fca404d6224fced36c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.131944 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.131833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/911c34c29a66f4fca404d6224fced36c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"911c34c29a66f4fca404d6224fced36c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.131944 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.131852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aba6b42d05a06c053c050c26af669cbb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" (UID: \"aba6b42d05a06c053c050c26af669cbb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.131944 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.131915 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/911c34c29a66f4fca404d6224fced36c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"911c34c29a66f4fca404d6224fced36c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.132089 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.131959 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/911c34c29a66f4fca404d6224fced36c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"911c34c29a66f4fca404d6224fced36c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.132089 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.132044 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aba6b42d05a06c053c050c26af669cbb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" (UID: \"aba6b42d05a06c053c050c26af669cbb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.155067 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.155044 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:36.255657 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.255581 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:36.324808 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.324768 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.329790 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.329594 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" Sep 19 04:15:36.356739 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.356673 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:36.457292 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.457252 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:36.557806 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.557733 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:36.658319 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.658288 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:36.675455 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.675429 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 19 04:15:36.675620 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.675598 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 19 04:15:36.693100 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.693079 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 04:15:36.758978 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.758955 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:36.764167 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.764095 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 04:10:35 +0000 UTC" deadline="2028-06-12 18:58:44.488448214 +0000 UTC" Sep 19 04:15:36.764267 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.764179 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15182h43m7.724279027s" Sep 19 04:15:36.859837 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.859754 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:36.900664 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:36.900621 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod911c34c29a66f4fca404d6224fced36c.slice/crio-f472f56a7c110f3b572a98c87abc64227394ec69be223adc4615c819a6b08fb1 WatchSource:0}: Error finding container f472f56a7c110f3b572a98c87abc64227394ec69be223adc4615c819a6b08fb1: Status 404 returned error can't find the container with id f472f56a7c110f3b572a98c87abc64227394ec69be223adc4615c819a6b08fb1 Sep 19 04:15:36.905685 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:36.905670 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 04:15:36.959392 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:36.959368 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba6b42d05a06c053c050c26af669cbb.slice/crio-ca80ede3624138358aeadf505e66231d4170236e55f743d9e875992ff04d3674 WatchSource:0}: Error finding container ca80ede3624138358aeadf505e66231d4170236e55f743d9e875992ff04d3674: Status 404 returned error can't find the container with id ca80ede3624138358aeadf505e66231d4170236e55f743d9e875992ff04d3674 Sep 19 04:15:36.959916 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:36.959899 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found" Sep 19 04:15:37.018271 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.018241 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 04:15:37.029885 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.029860 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" Sep 19 04:15:37.046232 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.046203 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 19 04:15:37.047236 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.047220 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" Sep 19 04:15:37.062366 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.062341 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 19 04:15:37.072763 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.072742 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 04:15:37.706990 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.706943 2577 apiserver.go:52] "Watching apiserver" Sep 19 04:15:37.714280 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.714251 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 19 04:15:37.714695 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.714667 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-8lnq4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8","openshift-cluster-node-tuning-operator/tuned-4c2t9","openshift-dns/node-resolver-rjjrv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal","openshift-multus/multus-hphtr","openshift-network-diagnostics/network-check-target-nqvp6","openshift-ovn-kubernetes/ovnkube-node-wkj2s","kube-system/konnectivity-agent-cnmp2","kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal","openshift-image-registry/node-ca-645z6","openshift-multus/multus-additional-cni-plugins-fr7lx","openshift-multus/network-metrics-daemon-mkdj2","openshift-network-operator/iptables-alerter-kspn4"] Sep 19 04:15:37.717584 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.717374 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:37.717584 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.717437 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:37.719673 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.719244 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.721689 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.721665 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.723339 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.723118 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 19 04:15:37.723339 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.723181 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dzcc4\"" Sep 19 04:15:37.723339 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.723215 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 19 04:15:37.723530 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.723409 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 19 04:15:37.723666 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.723645 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 19 04:15:37.724536 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.723792 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.724536 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.724051 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mlpth\"" Sep 19 04:15:37.724536 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.724359 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 19 04:15:37.726133 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.725886 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:15:37.727485 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.727467 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 19 04:15:37.727590 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.727475 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 19 04:15:37.728394 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.728085 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.729444 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.729032 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 19 04:15:37.729444 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.729145 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 19 04:15:37.729444 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.729196 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6r2x9\"" Sep 19 04:15:37.729444 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.729310 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-srp7b\"" Sep 19 04:15:37.729908 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.729768 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 19 04:15:37.730291 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.730221 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 19 04:15:37.730530 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.730511 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p5zp5\"" Sep 19 04:15:37.730728 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.730713 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 19 04:15:37.730892 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.730879 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 19 04:15:37.731614 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.731571 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.733461 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.733437 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.734219 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.734173 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 19 04:15:37.734524 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.734418 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wxs82\"" Sep 19 04:15:37.734914 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.734892 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 19 04:15:37.734990 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.734899 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 19 04:15:37.735348 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.735327 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 19 04:15:37.735448 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.735338 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 19 04:15:37.736457 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.736437 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 19 04:15:37.736708 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.736694 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 19 04:15:37.736750 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.736732 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:37.736851 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.736828 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.737515 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.737756 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gkqwc\"" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.737862 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.737872 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738427 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-run-netns\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738463 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7dfafbf3-d464-424a-96c9-ef104e8ede27-konnectivity-ca\") pod \"konnectivity-agent-cnmp2\" (UID: \"7dfafbf3-d464-424a-96c9-ef104e8ede27\") " pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738511 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-etc-selinux\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738539 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-var-lib-cni-bin\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738569 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd-host\") pod \"node-ca-645z6\" (UID: \"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd\") " pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-registration-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738643 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-device-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738671 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-cni-binary-copy\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738699 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac630975-bfc0-46ba-9195-78394cc767ea-tmp-dir\") pod \"node-resolver-rjjrv\" (UID: \"ac630975-bfc0-46ba-9195-78394cc767ea\") " pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738729 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-socket-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738751 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-sys\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738780 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7dfafbf3-d464-424a-96c9-ef104e8ede27-agent-certs\") pod \"konnectivity-agent-cnmp2\" (UID: \"7dfafbf3-d464-424a-96c9-ef104e8ede27\") " pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738809 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-cnibin\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738844 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-var-lib-cni-multus\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.739759 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738876 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-run-multus-certs\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738901 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738927 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd-serviceca\") pod \"node-ca-645z6\" (UID: \"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd\") " pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-sysconfig\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.738983 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-sysctl-conf\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739040 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-system-cni-dir\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng7mx\" (UniqueName: \"kubernetes.io/projected/c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd-kube-api-access-ng7mx\") pod \"node-ca-645z6\" (UID: \"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd\") " pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739094 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-systemd\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739122 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-var-lib-kubelet\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739179 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-host\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739216 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx7br\" (UniqueName: \"kubernetes.io/projected/ba92278e-f090-483f-adca-2d115b2932f0-kube-api-access-rx7br\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739239 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-conf-dir\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739267 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-daemon-config\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739299 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd52g\" (UniqueName: \"kubernetes.io/projected/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-kube-api-access-vd52g\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739828 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r89jr\" (UniqueName: \"kubernetes.io/projected/9409631a-115a-446a-b596-a3cdb932e9e8-kube-api-access-r89jr\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.740669 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-kubernetes\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739884 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-sysctl-d\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739913 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba92278e-f090-483f-adca-2d115b2932f0-etc-tuned\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739944 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-hostroot\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.739997 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjdbt\" (UniqueName: \"kubernetes.io/projected/ac630975-bfc0-46ba-9195-78394cc767ea-kube-api-access-sjdbt\") pod \"node-resolver-rjjrv\" (UID: \"ac630975-bfc0-46ba-9195-78394cc767ea\") " pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-sys-fs\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740060 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-run\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740092 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac630975-bfc0-46ba-9195-78394cc767ea-hosts-file\") pod \"node-resolver-rjjrv\" (UID: \"ac630975-bfc0-46ba-9195-78394cc767ea\") " pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740126 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-cni-dir\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-var-lib-kubelet\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-etc-kubernetes\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740337 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-modprobe-d\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740391 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-lib-modules\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740458 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba92278e-f090-483f-adca-2d115b2932f0-tmp\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740498 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-socket-dir-parent\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.740568 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-run-k8s-cni-cncf-io\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.741289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.741023 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.743753 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.743734 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:37.743971 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.743948 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:37.744627 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.744600 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hf26g\"" Sep 19 04:15:37.744711 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.744628 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 19 04:15:37.744913 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.744858 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 19 04:15:37.746102 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.746083 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.749301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.749283 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 19 04:15:37.749582 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.749565 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 19 04:15:37.749831 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.749811 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 19 04:15:37.749907 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.749883 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 19 04:15:37.750086 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.750066 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vr989\"" Sep 19 04:15:37.778133 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.778113 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jsr5g" Sep 19 04:15:37.787006 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.786986 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jsr5g" Sep 19 04:15:37.831341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.831305 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 19 04:15:37.841564 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841532 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-run-multus-certs\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.841713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-sysconfig\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.841713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841598 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-sysctl-conf\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.841713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a151acea-4940-48b8-b206-571d1ea4fb4f-ovnkube-script-lib\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.841713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841645 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-run-multus-certs\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.841713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841650 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-system-cni-dir\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.841713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng7mx\" (UniqueName: \"kubernetes.io/projected/c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd-kube-api-access-ng7mx\") pod \"node-ca-645z6\" (UID: \"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd\") " pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.841713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-host\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.841713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841715 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx7br\" (UniqueName: \"kubernetes.io/projected/ba92278e-f090-483f-adca-2d115b2932f0-kube-api-access-rx7br\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841737 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-kubelet\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841756 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-slash\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841789 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/26b9539c-8b72-4843-8ff6-7d72e51105bd-iptables-alerter-script\") pod \"iptables-alerter-kspn4\" (UID: \"26b9539c-8b72-4843-8ff6-7d72e51105bd\") " pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841799 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-sysctl-conf\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841815 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-conf-dir\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841840 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-daemon-config\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841845 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-sysconfig\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841872 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd52g\" (UniqueName: \"kubernetes.io/projected/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-kube-api-access-vd52g\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841896 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r89jr\" (UniqueName: \"kubernetes.io/projected/9409631a-115a-446a-b596-a3cdb932e9e8-kube-api-access-r89jr\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841916 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba92278e-f090-483f-adca-2d115b2932f0-etc-tuned\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlc98\" (UniqueName: \"kubernetes.io/projected/c9d8399c-1bc0-49da-8573-c6e428deebca-kube-api-access-tlc98\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841965 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-hostroot\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.841986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-sys-fs\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.842020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842010 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-run-systemd\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842061 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/74e989c5-7d7b-44b0-ba04-f5bc769838d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842084 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-cni-dir\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842106 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-etc-kubernetes\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842130 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-modprobe-d\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842240 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-etc-kubernetes\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842283 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-system-cni-dir\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842513 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-conf-dir\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842527 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 19 04:15:37.842550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842499 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-host\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842556 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-sys-fs\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842567 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-lib-modules\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842586 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba92278e-f090-483f-adca-2d115b2932f0-tmp\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-run-netns\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-socket-dir-parent\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-etc-selinux\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842676 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-etc-openvswitch\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842689 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-cni-bin\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842719 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74e989c5-7d7b-44b0-ba04-f5bc769838d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842736 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf4q4\" (UniqueName: \"kubernetes.io/projected/74e989c5-7d7b-44b0-ba04-f5bc769838d2-kube-api-access-rf4q4\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842753 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-var-lib-cni-bin\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842768 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-registration-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842865 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-run-openvswitch\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks7tr\" (UniqueName: \"kubernetes.io/projected/26b9539c-8b72-4843-8ff6-7d72e51105bd-kube-api-access-ks7tr\") pod \"iptables-alerter-kspn4\" (UID: \"26b9539c-8b72-4843-8ff6-7d72e51105bd\") " pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842902 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/74e989c5-7d7b-44b0-ba04-f5bc769838d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842938 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-cni-binary-copy\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.842981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842954 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-sys\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-run-ovn\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.842996 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7dfafbf3-d464-424a-96c9-ef104e8ede27-agent-certs\") pod \"konnectivity-agent-cnmp2\" (UID: \"7dfafbf3-d464-424a-96c9-ef104e8ede27\") " pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843020 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd-serviceca\") pod \"node-ca-645z6\" (UID: \"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd\") " pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843051 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-systemd-units\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-var-lib-openvswitch\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843081 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-daemon-config\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843105 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26b9539c-8b72-4843-8ff6-7d72e51105bd-host-slash\") pod \"iptables-alerter-kspn4\" (UID: \"26b9539c-8b72-4843-8ff6-7d72e51105bd\") " pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843132 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-kubernetes\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843188 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-cni-dir\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843199 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-sysctl-d\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843226 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjdbt\" (UniqueName: \"kubernetes.io/projected/ac630975-bfc0-46ba-9195-78394cc767ea-kube-api-access-sjdbt\") pod \"node-resolver-rjjrv\" (UID: \"ac630975-bfc0-46ba-9195-78394cc767ea\") " pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843409 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-run\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.843843 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843463 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf97s\" (UniqueName: \"kubernetes.io/projected/a151acea-4940-48b8-b206-571d1ea4fb4f-kube-api-access-pf97s\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843489 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4eea2ebc-5430-4f00-9ed7-492efe045fb0-dbus\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843513 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-multus-socket-dir-parent\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843514 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843547 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74e989c5-7d7b-44b0-ba04-f5bc769838d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac630975-bfc0-46ba-9195-78394cc767ea-hosts-file\") pod \"node-resolver-rjjrv\" (UID: \"ac630975-bfc0-46ba-9195-78394cc767ea\") " pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843584 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-var-lib-kubelet\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843600 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-node-log\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-run-k8s-cni-cncf-io\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-run-netns\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843659 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7dfafbf3-d464-424a-96c9-ef104e8ede27-konnectivity-ca\") pod \"konnectivity-agent-cnmp2\" (UID: \"7dfafbf3-d464-424a-96c9-ef104e8ede27\") " pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a151acea-4940-48b8-b206-571d1ea4fb4f-ovnkube-config\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a151acea-4940-48b8-b206-571d1ea4fb4f-ovn-node-metrics-cert\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843732 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd-host\") pod \"node-ca-645z6\" (UID: \"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd\") " pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-device-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843770 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-log-socket\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843793 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a151acea-4940-48b8-b206-571d1ea4fb4f-env-overrides\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.844635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843815 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74e989c5-7d7b-44b0-ba04-f5bc769838d2-system-cni-dir\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843831 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac630975-bfc0-46ba-9195-78394cc767ea-tmp-dir\") pod \"node-resolver-rjjrv\" (UID: \"ac630975-bfc0-46ba-9195-78394cc767ea\") " pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843864 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-socket-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843880 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-cni-netd\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843896 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4eea2ebc-5430-4f00-9ed7-492efe045fb0-kubelet-config\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843911 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74e989c5-7d7b-44b0-ba04-f5bc769838d2-cnibin\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843961 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-cni-binary-copy\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.843997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-systemd\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844016 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-var-lib-kubelet\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844018 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-sys\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-cnibin\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844047 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-var-lib-cni-multus\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844096 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-var-lib-cni-multus\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844124 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-var-lib-cni-bin\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844215 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-etc-selinux\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844248 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac630975-bfc0-46ba-9195-78394cc767ea-hosts-file\") pod \"node-resolver-rjjrv\" (UID: \"ac630975-bfc0-46ba-9195-78394cc767ea\") " pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844268 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-var-lib-kubelet\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844288 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-run-k8s-cni-cncf-io\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.845431 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844307 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-host-run-netns\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844696 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7dfafbf3-d464-424a-96c9-ef104e8ede27-konnectivity-ca\") pod \"konnectivity-agent-cnmp2\" (UID: \"7dfafbf3-d464-424a-96c9-ef104e8ede27\") " pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844752 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd-host\") pod \"node-ca-645z6\" (UID: \"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd\") " pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.844832 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-device-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845109 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-socket-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845123 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-hostroot\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845135 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-registration-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845215 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-var-lib-kubelet\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845226 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac630975-bfc0-46ba-9195-78394cc767ea-tmp-dir\") pod \"node-resolver-rjjrv\" (UID: \"ac630975-bfc0-46ba-9195-78394cc767ea\") " pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845249 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-lib-modules\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845299 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-modprobe-d\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845342 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-sysctl-d\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-cnibin\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845440 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-systemd\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845491 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-etc-kubernetes\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845503 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9409631a-115a-446a-b596-a3cdb932e9e8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845547 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba92278e-f090-483f-adca-2d115b2932f0-run\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.846287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.845804 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd-serviceca\") pod \"node-ca-645z6\" (UID: \"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd\") " pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.847061 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.846350 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba92278e-f090-483f-adca-2d115b2932f0-etc-tuned\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.847326 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.847305 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7dfafbf3-d464-424a-96c9-ef104e8ede27-agent-certs\") pod \"konnectivity-agent-cnmp2\" (UID: \"7dfafbf3-d464-424a-96c9-ef104e8ede27\") " pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:15:37.851350 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.851328 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba92278e-f090-483f-adca-2d115b2932f0-tmp\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.860018 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.859994 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng7mx\" (UniqueName: \"kubernetes.io/projected/c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd-kube-api-access-ng7mx\") pod \"node-ca-645z6\" (UID: \"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd\") " pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:37.860322 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.860292 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx7br\" (UniqueName: \"kubernetes.io/projected/ba92278e-f090-483f-adca-2d115b2932f0-kube-api-access-rx7br\") pod \"tuned-4c2t9\" (UID: \"ba92278e-f090-483f-adca-2d115b2932f0\") " pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:37.861214 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.861139 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r89jr\" (UniqueName: \"kubernetes.io/projected/9409631a-115a-446a-b596-a3cdb932e9e8-kube-api-access-r89jr\") pod \"aws-ebs-csi-driver-node-jtgl8\" (UID: \"9409631a-115a-446a-b596-a3cdb932e9e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:37.861663 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.861633 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd52g\" (UniqueName: \"kubernetes.io/projected/3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4-kube-api-access-vd52g\") pod \"multus-hphtr\" (UID: \"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4\") " pod="openshift-multus/multus-hphtr" Sep 19 04:15:37.863608 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.863586 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 04:15:37.863708 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.863614 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 04:15:37.863708 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.863627 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dd4bt for pod openshift-network-diagnostics/network-check-target-nqvp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:37.863708 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.863686 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt podName:f184eae1-445c-4516-982f-908ad6ed1dda nodeName:}" failed. No retries permitted until 2026-09-19 04:15:38.36366631 +0000 UTC m=+4.684764448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dd4bt" (UniqueName: "kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt") pod "network-check-target-nqvp6" (UID: "f184eae1-445c-4516-982f-908ad6ed1dda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:37.865387 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.865362 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjdbt\" (UniqueName: \"kubernetes.io/projected/ac630975-bfc0-46ba-9195-78394cc767ea-kube-api-access-sjdbt\") pod \"node-resolver-rjjrv\" (UID: \"ac630975-bfc0-46ba-9195-78394cc767ea\") " pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:37.893700 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.893621 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" event={"ID":"aba6b42d05a06c053c050c26af669cbb","Type":"ContainerStarted","Data":"ca80ede3624138358aeadf505e66231d4170236e55f743d9e875992ff04d3674"} Sep 19 04:15:37.895141 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.895115 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" event={"ID":"911c34c29a66f4fca404d6224fced36c","Type":"ContainerStarted","Data":"f472f56a7c110f3b572a98c87abc64227394ec69be223adc4615c819a6b08fb1"} Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945257 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945360 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945395 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf97s\" (UniqueName: \"kubernetes.io/projected/a151acea-4940-48b8-b206-571d1ea4fb4f-kube-api-access-pf97s\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.945413 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.945491 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs podName:c9d8399c-1bc0-49da-8573-c6e428deebca nodeName:}" failed. No retries permitted until 2026-09-19 04:15:38.445472258 +0000 UTC m=+4.766570397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs") pod "network-metrics-daemon-mkdj2" (UID: "c9d8399c-1bc0-49da-8573-c6e428deebca") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945562 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4eea2ebc-5430-4f00-9ed7-492efe045fb0-dbus\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945418 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4eea2ebc-5430-4f00-9ed7-492efe045fb0-dbus\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945605 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945630 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74e989c5-7d7b-44b0-ba04-f5bc769838d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945655 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-node-log\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945699 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-node-log\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.945723 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:37.945765 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret podName:4eea2ebc-5430-4f00-9ed7-492efe045fb0 nodeName:}" failed. No retries permitted until 2026-09-19 04:15:38.445751887 +0000 UTC m=+4.766850016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret") pod "global-pull-secret-syncer-8lnq4" (UID: "4eea2ebc-5430-4f00-9ed7-492efe045fb0") : object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945794 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a151acea-4940-48b8-b206-571d1ea4fb4f-ovnkube-config\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945824 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a151acea-4940-48b8-b206-571d1ea4fb4f-ovn-node-metrics-cert\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.946169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-log-socket\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945854 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74e989c5-7d7b-44b0-ba04-f5bc769838d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945894 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-log-socket\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945919 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a151acea-4940-48b8-b206-571d1ea4fb4f-env-overrides\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74e989c5-7d7b-44b0-ba04-f5bc769838d2-system-cni-dir\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945969 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-cni-netd\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.945993 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4eea2ebc-5430-4f00-9ed7-492efe045fb0-kubelet-config\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74e989c5-7d7b-44b0-ba04-f5bc769838d2-cnibin\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946070 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74e989c5-7d7b-44b0-ba04-f5bc769838d2-system-cni-dir\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946077 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a151acea-4940-48b8-b206-571d1ea4fb4f-ovnkube-script-lib\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946041 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4eea2ebc-5430-4f00-9ed7-492efe045fb0-kubelet-config\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-kubelet\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946074 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-cni-netd\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946112 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74e989c5-7d7b-44b0-ba04-f5bc769838d2-cnibin\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946185 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-kubelet\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946216 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-slash\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946245 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/26b9539c-8b72-4843-8ff6-7d72e51105bd-iptables-alerter-script\") pod \"iptables-alerter-kspn4\" (UID: \"26b9539c-8b72-4843-8ff6-7d72e51105bd\") " pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.947004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946275 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlc98\" (UniqueName: \"kubernetes.io/projected/c9d8399c-1bc0-49da-8573-c6e428deebca-kube-api-access-tlc98\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-run-systemd\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946327 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946347 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/74e989c5-7d7b-44b0-ba04-f5bc769838d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946366 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-run-netns\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946383 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-etc-openvswitch\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946396 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-cni-bin\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74e989c5-7d7b-44b0-ba04-f5bc769838d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946425 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf4q4\" (UniqueName: \"kubernetes.io/projected/74e989c5-7d7b-44b0-ba04-f5bc769838d2-kube-api-access-rf4q4\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946441 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-run-openvswitch\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks7tr\" (UniqueName: \"kubernetes.io/projected/26b9539c-8b72-4843-8ff6-7d72e51105bd-kube-api-access-ks7tr\") pod \"iptables-alerter-kspn4\" (UID: \"26b9539c-8b72-4843-8ff6-7d72e51105bd\") " pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946466 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a151acea-4940-48b8-b206-571d1ea4fb4f-env-overrides\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946471 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/74e989c5-7d7b-44b0-ba04-f5bc769838d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946514 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-run-ovn\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-systemd-units\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946558 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-var-lib-openvswitch\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26b9539c-8b72-4843-8ff6-7d72e51105bd-host-slash\") pod \"iptables-alerter-kspn4\" (UID: \"26b9539c-8b72-4843-8ff6-7d72e51105bd\") " pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.947592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946624 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26b9539c-8b72-4843-8ff6-7d72e51105bd-host-slash\") pod \"iptables-alerter-kspn4\" (UID: \"26b9539c-8b72-4843-8ff6-7d72e51105bd\") " pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946648 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-run-ovn\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946651 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a151acea-4940-48b8-b206-571d1ea4fb4f-ovnkube-script-lib\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946669 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-systemd-units\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946692 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-var-lib-openvswitch\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946718 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-slash\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946747 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946756 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-run-systemd\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946721 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-etc-openvswitch\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946794 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-run-netns\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946805 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/26b9539c-8b72-4843-8ff6-7d72e51105bd-iptables-alerter-script\") pod \"iptables-alerter-kspn4\" (UID: \"26b9539c-8b72-4843-8ff6-7d72e51105bd\") " pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946848 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/74e989c5-7d7b-44b0-ba04-f5bc769838d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946905 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-run-openvswitch\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.946931 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a151acea-4940-48b8-b206-571d1ea4fb4f-host-cni-bin\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.947222 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/74e989c5-7d7b-44b0-ba04-f5bc769838d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.947286 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74e989c5-7d7b-44b0-ba04-f5bc769838d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:37.948312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.947559 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a151acea-4940-48b8-b206-571d1ea4fb4f-ovnkube-config\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.949643 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.949624 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a151acea-4940-48b8-b206-571d1ea4fb4f-ovn-node-metrics-cert\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.966469 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.966444 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf97s\" (UniqueName: \"kubernetes.io/projected/a151acea-4940-48b8-b206-571d1ea4fb4f-kube-api-access-pf97s\") pod \"ovnkube-node-wkj2s\" (UID: \"a151acea-4940-48b8-b206-571d1ea4fb4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:37.966469 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.966458 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlc98\" (UniqueName: \"kubernetes.io/projected/c9d8399c-1bc0-49da-8573-c6e428deebca-kube-api-access-tlc98\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:37.966655 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.966592 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks7tr\" (UniqueName: \"kubernetes.io/projected/26b9539c-8b72-4843-8ff6-7d72e51105bd-kube-api-access-ks7tr\") pod \"iptables-alerter-kspn4\" (UID: \"26b9539c-8b72-4843-8ff6-7d72e51105bd\") " pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:37.967004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:37.966985 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf4q4\" (UniqueName: \"kubernetes.io/projected/74e989c5-7d7b-44b0-ba04-f5bc769838d2-kube-api-access-rf4q4\") pod \"multus-additional-cni-plugins-fr7lx\" (UID: \"74e989c5-7d7b-44b0-ba04-f5bc769838d2\") " pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:38.038365 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.038281 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" Sep 19 04:15:38.048433 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.048408 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" Sep 19 04:15:38.058402 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.058378 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-645z6" Sep 19 04:15:38.064138 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.064115 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:15:38.070859 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.070838 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rjjrv" Sep 19 04:15:38.080538 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.080512 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hphtr" Sep 19 04:15:38.087314 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.087295 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:15:38.095860 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.095839 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" Sep 19 04:15:38.103500 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.103483 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kspn4" Sep 19 04:15:38.183189 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.183146 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 04:15:38.450171 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.450066 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:38.450171 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.450117 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:38.450171 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.450147 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:38.450440 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:38.450256 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:38.450440 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:38.450256 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:38.450440 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:38.450268 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 04:15:38.450440 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:38.450293 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 04:15:38.450440 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:38.450303 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dd4bt for pod openshift-network-diagnostics/network-check-target-nqvp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:38.450440 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:38.450316 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret podName:4eea2ebc-5430-4f00-9ed7-492efe045fb0 nodeName:}" failed. No retries permitted until 2026-09-19 04:15:39.450299488 +0000 UTC m=+5.771397615 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret") pod "global-pull-secret-syncer-8lnq4" (UID: "4eea2ebc-5430-4f00-9ed7-492efe045fb0") : object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:38.450440 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:38.450335 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs podName:c9d8399c-1bc0-49da-8573-c6e428deebca nodeName:}" failed. No retries permitted until 2026-09-19 04:15:39.450326168 +0000 UTC m=+5.771424297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs") pod "network-metrics-daemon-mkdj2" (UID: "c9d8399c-1bc0-49da-8573-c6e428deebca") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:38.450440 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:38.450348 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt podName:f184eae1-445c-4516-982f-908ad6ed1dda nodeName:}" failed. No retries permitted until 2026-09-19 04:15:39.450340606 +0000 UTC m=+5.771438729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dd4bt" (UniqueName: "kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt") pod "network-check-target-nqvp6" (UID: "f184eae1-445c-4516-982f-908ad6ed1dda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:38.790767 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.790689 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 04:10:37 +0000 UTC" deadline="2028-04-28 12:15:40.421737824 +0000 UTC" Sep 19 04:15:38.790767 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.790715 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14096h0m1.631025233s" Sep 19 04:15:38.888475 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:38.888437 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:38.888652 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:38.888554 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:39.001102 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:39.001066 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac630975_bfc0_46ba_9195_78394cc767ea.slice/crio-f050e51a9fee617654160f58cb981dcc3b6f5b40521e4e3ce7c6f618cf45c9cc WatchSource:0}: Error finding container f050e51a9fee617654160f58cb981dcc3b6f5b40521e4e3ce7c6f618cf45c9cc: Status 404 returned error can't find the container with id f050e51a9fee617654160f58cb981dcc3b6f5b40521e4e3ce7c6f618cf45c9cc Sep 19 04:15:39.003138 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:39.003109 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda151acea_4940_48b8_b206_571d1ea4fb4f.slice/crio-9c5e40e63eab1b3db21fc044f7aca2e835a2c62ccc58b2c9beac261aaf6ab215 WatchSource:0}: Error finding container 9c5e40e63eab1b3db21fc044f7aca2e835a2c62ccc58b2c9beac261aaf6ab215: Status 404 returned error can't find the container with id 9c5e40e63eab1b3db21fc044f7aca2e835a2c62ccc58b2c9beac261aaf6ab215 Sep 19 04:15:39.004803 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:39.004745 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e989c5_7d7b_44b0_ba04_f5bc769838d2.slice/crio-eb875b776667f6bda93294baa8567c28aeb17448f039eda1235985079e8b9b8a WatchSource:0}: Error finding container eb875b776667f6bda93294baa8567c28aeb17448f039eda1235985079e8b9b8a: Status 404 returned error can't find the container with id eb875b776667f6bda93294baa8567c28aeb17448f039eda1235985079e8b9b8a Sep 19 04:15:39.005606 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:39.005579 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d3a01e_11a7_42ec_9ec0_72d1c5b002dd.slice/crio-920c78163be0e6b6b63ef7831aa6b4a3ebdc6827e9d90179de7af5b79cba75d2 WatchSource:0}: Error finding container 920c78163be0e6b6b63ef7831aa6b4a3ebdc6827e9d90179de7af5b79cba75d2: Status 404 returned error can't find the container with id 920c78163be0e6b6b63ef7831aa6b4a3ebdc6827e9d90179de7af5b79cba75d2 Sep 19 04:15:39.006374 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:39.006348 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba92278e_f090_483f_adca_2d115b2932f0.slice/crio-149c0f9dd364342a8938bc297306509e85615e165de721985013d6d417aad600 WatchSource:0}: Error finding container 149c0f9dd364342a8938bc297306509e85615e165de721985013d6d417aad600: Status 404 returned error can't find the container with id 149c0f9dd364342a8938bc297306509e85615e165de721985013d6d417aad600 Sep 19 04:15:39.008741 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:39.008459 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dfafbf3_d464_424a_96c9_ef104e8ede27.slice/crio-4580f73269674019ef89b33c8127d1d54c450dff3236868f06d83f6a842b0a1f WatchSource:0}: Error finding container 4580f73269674019ef89b33c8127d1d54c450dff3236868f06d83f6a842b0a1f: Status 404 returned error can't find the container with id 4580f73269674019ef89b33c8127d1d54c450dff3236868f06d83f6a842b0a1f Sep 19 04:15:39.009124 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:15:39.009107 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b9539c_8b72_4843_8ff6_7d72e51105bd.slice/crio-fcfd560d5f062ea6114b080392040bf3104f9c42691fbf50c0d2030420576a14 WatchSource:0}: Error finding container fcfd560d5f062ea6114b080392040bf3104f9c42691fbf50c0d2030420576a14: Status 404 returned error can't find the container with id fcfd560d5f062ea6114b080392040bf3104f9c42691fbf50c0d2030420576a14 Sep 19 04:15:39.457560 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.457284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:39.457560 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.457536 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:39.457560 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.457563 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:39.457818 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.457475 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 04:15:39.457818 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.457616 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 04:15:39.457818 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.457633 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dd4bt for pod openshift-network-diagnostics/network-check-target-nqvp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:39.457818 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.457665 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:39.457818 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.457683 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:39.457818 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.457703 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt podName:f184eae1-445c-4516-982f-908ad6ed1dda nodeName:}" failed. No retries permitted until 2026-09-19 04:15:41.457685271 +0000 UTC m=+7.778783398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dd4bt" (UniqueName: "kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt") pod "network-check-target-nqvp6" (UID: "f184eae1-445c-4516-982f-908ad6ed1dda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:39.457818 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.457723 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret podName:4eea2ebc-5430-4f00-9ed7-492efe045fb0 nodeName:}" failed. No retries permitted until 2026-09-19 04:15:41.457714179 +0000 UTC m=+7.778812303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret") pod "global-pull-secret-syncer-8lnq4" (UID: "4eea2ebc-5430-4f00-9ed7-492efe045fb0") : object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:39.457818 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.457747 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs podName:c9d8399c-1bc0-49da-8573-c6e428deebca nodeName:}" failed. No retries permitted until 2026-09-19 04:15:41.457727503 +0000 UTC m=+7.778825647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs") pod "network-metrics-daemon-mkdj2" (UID: "c9d8399c-1bc0-49da-8573-c6e428deebca") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:39.791723 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.791571 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 04:10:37 +0000 UTC" deadline="2028-05-15 09:28:30.24958262 +0000 UTC" Sep 19 04:15:39.791723 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.791678 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14501h12m50.457909227s" Sep 19 04:15:39.890647 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.890615 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:39.890827 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.890758 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:39.891361 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.891341 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:39.891479 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:39.891458 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:39.906990 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.906804 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" event={"ID":"aba6b42d05a06c053c050c26af669cbb","Type":"ContainerStarted","Data":"95a92e6342ff096c79df582455ed07c614f5e5f6165d77fbf6155cd5d27f811f"} Sep 19 04:15:39.914953 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.914069 2577 generic.go:356] "Generic (PLEG): container finished" podID="911c34c29a66f4fca404d6224fced36c" containerID="3c34cbe13cc2af8ad239564416208d0421d0ad2e2af35ea789e40d9f5c080632" exitCode=0 Sep 19 04:15:39.914953 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.914175 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" event={"ID":"911c34c29a66f4fca404d6224fced36c","Type":"ContainerDied","Data":"3c34cbe13cc2af8ad239564416208d0421d0ad2e2af35ea789e40d9f5c080632"} Sep 19 04:15:39.927650 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.927529 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerStarted","Data":"9c5e40e63eab1b3db21fc044f7aca2e835a2c62ccc58b2c9beac261aaf6ab215"} Sep 19 04:15:39.932355 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.932325 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rjjrv" event={"ID":"ac630975-bfc0-46ba-9195-78394cc767ea","Type":"ContainerStarted","Data":"f050e51a9fee617654160f58cb981dcc3b6f5b40521e4e3ce7c6f618cf45c9cc"} Sep 19 04:15:39.934809 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.934776 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" event={"ID":"9409631a-115a-446a-b596-a3cdb932e9e8","Type":"ContainerStarted","Data":"eac95ec95ad4123e9d902c7dc1e12cbb490b1f419301d15757c1b60fd5dd943d"} Sep 19 04:15:39.944050 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.944015 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kspn4" event={"ID":"26b9539c-8b72-4843-8ff6-7d72e51105bd","Type":"ContainerStarted","Data":"fcfd560d5f062ea6114b080392040bf3104f9c42691fbf50c0d2030420576a14"} Sep 19 04:15:39.947941 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.947883 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" podStartSLOduration=2.947865345 podStartE2EDuration="2.947865345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:15:39.92491883 +0000 UTC m=+6.246016978" watchObservedRunningTime="2026-09-19 04:15:39.947865345 +0000 UTC m=+6.268963494" Sep 19 04:15:39.951438 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.951392 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hphtr" event={"ID":"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4","Type":"ContainerStarted","Data":"bc5781ffac17358582a1e972122583762e6a57e5c98a9a4837a643207b6a7ddc"} Sep 19 04:15:39.956787 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.956756 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cnmp2" event={"ID":"7dfafbf3-d464-424a-96c9-ef104e8ede27","Type":"ContainerStarted","Data":"4580f73269674019ef89b33c8127d1d54c450dff3236868f06d83f6a842b0a1f"} Sep 19 04:15:39.971637 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.971568 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" event={"ID":"ba92278e-f090-483f-adca-2d115b2932f0","Type":"ContainerStarted","Data":"149c0f9dd364342a8938bc297306509e85615e165de721985013d6d417aad600"} Sep 19 04:15:39.978810 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.978721 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-645z6" event={"ID":"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd","Type":"ContainerStarted","Data":"920c78163be0e6b6b63ef7831aa6b4a3ebdc6827e9d90179de7af5b79cba75d2"} Sep 19 04:15:39.988300 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:39.988214 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" event={"ID":"74e989c5-7d7b-44b0-ba04-f5bc769838d2","Type":"ContainerStarted","Data":"eb875b776667f6bda93294baa8567c28aeb17448f039eda1235985079e8b9b8a"} Sep 19 04:15:40.889187 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:40.888688 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:40.889187 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:40.888819 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:41.032952 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:41.032907 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" event={"ID":"911c34c29a66f4fca404d6224fced36c","Type":"ContainerStarted","Data":"f91d3416db7447f02234054c08e55208df02e8b098ec14d86a6b9b33e3ce809d"} Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:41.475658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:41.475719 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:41.475748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.475915 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.475983 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret podName:4eea2ebc-5430-4f00-9ed7-492efe045fb0 nodeName:}" failed. No retries permitted until 2026-09-19 04:15:45.475962216 +0000 UTC m=+11.797060364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret") pod "global-pull-secret-syncer-8lnq4" (UID: "4eea2ebc-5430-4f00-9ed7-492efe045fb0") : object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.476075 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.476090 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.476101 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dd4bt for pod openshift-network-diagnostics/network-check-target-nqvp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.476139 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt podName:f184eae1-445c-4516-982f-908ad6ed1dda nodeName:}" failed. No retries permitted until 2026-09-19 04:15:45.47612764 +0000 UTC m=+11.797225778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dd4bt" (UniqueName: "kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt") pod "network-check-target-nqvp6" (UID: "f184eae1-445c-4516-982f-908ad6ed1dda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.476218 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:41.476605 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.476249 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs podName:c9d8399c-1bc0-49da-8573-c6e428deebca nodeName:}" failed. No retries permitted until 2026-09-19 04:15:45.476237994 +0000 UTC m=+11.797336120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs") pod "network-metrics-daemon-mkdj2" (UID: "c9d8399c-1bc0-49da-8573-c6e428deebca") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:41.889004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:41.888321 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:41.889004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:41.888373 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:41.889004 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.888479 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:41.889004 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:41.888615 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:42.888320 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:42.888278 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:42.888502 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:42.888413 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:43.888101 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:43.888047 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:43.888101 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:43.888086 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:43.888651 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:43.888218 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:43.888651 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:43.888345 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:44.888571 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:44.888034 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:44.888571 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:44.888191 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:45.510037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:45.510091 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:45.510177 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.510328 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.510351 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.510364 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dd4bt for pod openshift-network-diagnostics/network-check-target-nqvp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.510420 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt podName:f184eae1-445c-4516-982f-908ad6ed1dda nodeName:}" failed. No retries permitted until 2026-09-19 04:15:53.510402202 +0000 UTC m=+19.831500341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dd4bt" (UniqueName: "kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt") pod "network-check-target-nqvp6" (UID: "f184eae1-445c-4516-982f-908ad6ed1dda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.510488 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.510520 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs podName:c9d8399c-1bc0-49da-8573-c6e428deebca nodeName:}" failed. No retries permitted until 2026-09-19 04:15:53.510508738 +0000 UTC m=+19.831606877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs") pod "network-metrics-daemon-mkdj2" (UID: "c9d8399c-1bc0-49da-8573-c6e428deebca") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.510584 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:45.510951 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.510612 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret podName:4eea2ebc-5430-4f00-9ed7-492efe045fb0 nodeName:}" failed. No retries permitted until 2026-09-19 04:15:53.510602507 +0000 UTC m=+19.831700635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret") pod "global-pull-secret-syncer-8lnq4" (UID: "4eea2ebc-5430-4f00-9ed7-492efe045fb0") : object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:45.889421 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:45.888976 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:45.889421 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:45.889043 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:45.889421 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.889171 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:45.889421 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:45.889319 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:46.888517 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:46.888480 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:46.888806 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:46.888611 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:47.888215 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:47.888184 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:47.888640 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:47.888324 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:47.888726 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:47.888702 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:47.888822 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:47.888799 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:48.888216 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:48.888178 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:48.888359 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:48.888323 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:49.888714 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:49.888667 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:49.889183 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:49.888672 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:49.889183 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:49.888802 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:49.889183 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:49.888892 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:50.888495 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:50.888459 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:50.888673 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:50.888578 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:51.888271 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:51.888227 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:51.888718 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:51.888289 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:51.888718 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:51.888377 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:51.888718 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:51.888514 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:52.888232 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:52.888198 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:52.888416 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:52.888338 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:53.574444 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:53.574396 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:53.574648 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:53.574459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:53.574648 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:53.574488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:53.574648 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.574623 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 04:15:53.574798 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.574661 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 04:15:53.574798 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.574668 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:53.574798 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.574667 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:53.574798 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.574732 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs podName:c9d8399c-1bc0-49da-8573-c6e428deebca nodeName:}" failed. No retries permitted until 2026-09-19 04:16:09.57471315 +0000 UTC m=+35.895811273 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs") pod "network-metrics-daemon-mkdj2" (UID: "c9d8399c-1bc0-49da-8573-c6e428deebca") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:15:53.574798 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.574750 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret podName:4eea2ebc-5430-4f00-9ed7-492efe045fb0 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:09.574741116 +0000 UTC m=+35.895839243 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret") pod "global-pull-secret-syncer-8lnq4" (UID: "4eea2ebc-5430-4f00-9ed7-492efe045fb0") : object "kube-system"/"original-pull-secret" not registered Sep 19 04:15:53.574798 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.574676 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dd4bt for pod openshift-network-diagnostics/network-check-target-nqvp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:53.575017 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.574804 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt podName:f184eae1-445c-4516-982f-908ad6ed1dda nodeName:}" failed. No retries permitted until 2026-09-19 04:16:09.574794096 +0000 UTC m=+35.895892221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dd4bt" (UniqueName: "kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt") pod "network-check-target-nqvp6" (UID: "f184eae1-445c-4516-982f-908ad6ed1dda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:15:53.889256 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:53.889145 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:53.889696 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.889349 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:53.889696 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:53.889499 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:53.889696 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:53.889647 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:54.888442 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:54.888400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:54.888624 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:54.888534 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:55.889075 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:55.889037 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:55.889534 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:55.889172 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:55.889534 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:55.889187 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:55.889534 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:55.889314 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:56.887961 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:56.887928 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:56.888111 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:56.888070 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:57.888561 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:57.888202 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:57.889472 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:57.888269 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:57.889472 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:57.888646 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:15:57.889472 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:57.888731 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:58.068257 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.068232 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 19 04:15:58.071017 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.070992 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hphtr" event={"ID":"3cb36ca0-94e2-4ce7-abbb-dec97b78c0b4","Type":"ContainerStarted","Data":"c68cf3d91c80127c72b75e88515f92f9ed3ea0dc33315cf3da75b293b1689ae6"} Sep 19 04:15:58.072374 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.072344 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cnmp2" event={"ID":"7dfafbf3-d464-424a-96c9-ef104e8ede27","Type":"ContainerStarted","Data":"038229a84c56dc5cda7569819ddcd52a0893f4e03cd33564429658bbf1ff4f8b"} Sep 19 04:15:58.073896 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.073870 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" event={"ID":"ba92278e-f090-483f-adca-2d115b2932f0","Type":"ContainerStarted","Data":"a49de648afc6adf3aae03295558e267badb8b9f357fb08b31b4a5748a43084d0"} Sep 19 04:15:58.079557 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.078569 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-645z6" event={"ID":"c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd","Type":"ContainerStarted","Data":"fe5b30a10d71ebc47aaad806a86ae6521ca28973ce86509b076dc7d328027dfb"} Sep 19 04:15:58.080935 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.080913 2577 generic.go:356] "Generic (PLEG): container finished" podID="74e989c5-7d7b-44b0-ba04-f5bc769838d2" containerID="02cf1dc351615d0c70bfb94a89446ed79ab6cc2f840afafcbc0a7362aa277536" exitCode=0 Sep 19 04:15:58.081013 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.080994 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" event={"ID":"74e989c5-7d7b-44b0-ba04-f5bc769838d2","Type":"ContainerDied","Data":"02cf1dc351615d0c70bfb94a89446ed79ab6cc2f840afafcbc0a7362aa277536"} Sep 19 04:15:58.083812 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.083796 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wkj2s_a151acea-4940-48b8-b206-571d1ea4fb4f/ovn-acl-logging/0.log" Sep 19 04:15:58.084126 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.084110 2577 generic.go:356] "Generic (PLEG): container finished" podID="a151acea-4940-48b8-b206-571d1ea4fb4f" containerID="b20c99dc7219fecd1e72e43e0aa9f54211c2bf53cac8250c9204258c732ce250" exitCode=1 Sep 19 04:15:58.084206 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.084188 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerStarted","Data":"35cd5ff325c3931bce66b0081086167de4b43f0f55e269ea3bee0688cdc32313"} Sep 19 04:15:58.084266 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.084252 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerStarted","Data":"ecc47bc880df876345c82cff9810751fa8206c0f804cf49bba1e0f02be72525a"} Sep 19 04:15:58.084307 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.084296 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerStarted","Data":"4102b4cbd7353df6d08be93a0ea7349dba4429046720f104830a68edf1e5e868"} Sep 19 04:15:58.084346 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.084329 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerStarted","Data":"baa3d8912f64852c728bf196ded76f8a76e632d41d8a45504fd04e11c477241d"} Sep 19 04:15:58.084377 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.084364 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerDied","Data":"b20c99dc7219fecd1e72e43e0aa9f54211c2bf53cac8250c9204258c732ce250"} Sep 19 04:15:58.084423 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.084411 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerStarted","Data":"0b200400b68e230b5df9db69c3ea01d624282194868f1afd8654c45bbddee2f4"} Sep 19 04:15:58.085520 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.085502 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rjjrv" event={"ID":"ac630975-bfc0-46ba-9195-78394cc767ea","Type":"ContainerStarted","Data":"eb00dc04a8d508dcb1cdae35b5c85c8a3efade63b283bbf20b3ee0a4dc431da2"} Sep 19 04:15:58.087117 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.087099 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" event={"ID":"9409631a-115a-446a-b596-a3cdb932e9e8","Type":"ContainerStarted","Data":"41094ef5c759127b8ddd167fbbbc4a6d3ddbf39367db9ff44d875b78b46c8a1e"} Sep 19 04:15:58.087210 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.087145 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" event={"ID":"9409631a-115a-446a-b596-a3cdb932e9e8","Type":"ContainerStarted","Data":"2034faa4a2b2b47b39700a77b770a77948e4c659ea3cda45fb7266c3ab052169"} Sep 19 04:15:58.095855 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.095806 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" podStartSLOduration=21.095788282 podStartE2EDuration="21.095788282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:15:41.05853551 +0000 UTC m=+7.379633682" watchObservedRunningTime="2026-09-19 04:15:58.095788282 +0000 UTC m=+24.416886429" Sep 19 04:15:58.095976 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.095918 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hphtr" podStartSLOduration=4.105262398 podStartE2EDuration="22.095904385s" totalImagesPullingTime="17.990641987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:15:58.092883915 +0000 UTC m=+24.413982061" watchObservedRunningTime="2026-09-19 04:15:58.095904385 +0000 UTC m=+24.417002548" Sep 19 04:15:58.154847 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.154748 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-645z6" podStartSLOduration=5.194182531 podStartE2EDuration="23.154732417s" totalImagesPullingTime="17.960549886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:15:58.154247885 +0000 UTC m=+24.475346030" watchObservedRunningTime="2026-09-19 04:15:58.154732417 +0000 UTC m=+24.475830562" Sep 19 04:15:58.155042 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.155023 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4c2t9" podStartSLOduration=5.196931303 podStartE2EDuration="23.155016409s" totalImagesPullingTime="17.958085106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:15:58.12557827 +0000 UTC m=+24.446676416" watchObservedRunningTime="2026-09-19 04:15:58.155016409 +0000 UTC m=+24.476114554" Sep 19 04:15:58.183080 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.182997 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cnmp2" podStartSLOduration=9.532657657 podStartE2EDuration="23.182981992s" totalImagesPullingTime="13.650324335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:15:58.182057691 +0000 UTC m=+24.503155836" watchObservedRunningTime="2026-09-19 04:15:58.182981992 +0000 UTC m=+24.504080137" Sep 19 04:15:58.202458 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.202420 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rjjrv" podStartSLOduration=4.239560102 podStartE2EDuration="22.202405589s" totalImagesPullingTime="17.962845487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:15:58.20164091 +0000 UTC m=+24.522739055" watchObservedRunningTime="2026-09-19 04:15:58.202405589 +0000 UTC m=+24.523503732" Sep 19 04:15:58.294730 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.294697 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:15:58.377907 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.377877 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:15:58.830059 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.829936 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T04:15:58.068253955Z","UUID":"217db76d-3e93-4ca8-bbd5-7b22dcc79b94","Handler":null,"Name":"","Endpoint":""} Sep 19 04:15:58.831678 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.831655 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 19 04:15:58.831781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.831712 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 19 04:15:58.888472 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:58.888449 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:15:58.888566 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:58.888548 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:15:59.092080 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:59.091992 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" event={"ID":"9409631a-115a-446a-b596-a3cdb932e9e8","Type":"ContainerStarted","Data":"8beb6a2c8a812f07f25848cfd4d83b95a1837d86defc5f0cbd3f89d84cac0216"} Sep 19 04:15:59.094436 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:59.094411 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kspn4" event={"ID":"26b9539c-8b72-4843-8ff6-7d72e51105bd","Type":"ContainerStarted","Data":"ccbbb283a471f731c18a0876f9af8d627b8802935221940fd931737ed1fe82d2"} Sep 19 04:15:59.116139 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:59.116077 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jtgl8" podStartSLOduration=4.675621785 podStartE2EDuration="24.11605856s" totalImagesPullingTime="19.440436775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:15:59.115700987 +0000 UTC m=+25.436799134" watchObservedRunningTime="2026-09-19 04:15:59.11605856 +0000 UTC m=+25.437156705" Sep 19 04:15:59.136999 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:59.136943 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kspn4" podStartSLOduration=5.182751601 podStartE2EDuration="23.136923431s" totalImagesPullingTime="17.95417183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:15:59.136687071 +0000 UTC m=+25.457785212" watchObservedRunningTime="2026-09-19 04:15:59.136923431 +0000 UTC m=+25.458021579" Sep 19 04:15:59.282754 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:59.282722 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:15:59.361072 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:59.360985 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:15:59.888203 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:59.887924 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:15:59.888389 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:59.888315 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:15:59.888389 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:15:59.887994 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:15:59.888516 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:15:59.888488 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:16:00.099996 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:00.099969 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wkj2s_a151acea-4940-48b8-b206-571d1ea4fb4f/ovn-acl-logging/0.log" Sep 19 04:16:00.100506 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:00.100478 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerStarted","Data":"9ccf4eb22df9467258c068d4ce49c8c388c9abd96381391c119a5e76c6c4f406"} Sep 19 04:16:00.662432 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:00.662393 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:16:00.662993 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:00.662971 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:16:00.888358 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:00.888326 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:00.888515 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:00.888438 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:16:01.103396 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:01.103368 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:16:01.103905 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:01.103859 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cnmp2" Sep 19 04:16:01.228196 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:01.228138 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:01.888022 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:01.887988 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:01.888022 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:01.888027 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:01.888286 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:01.888128 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:16:01.888286 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:01.888245 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:16:02.888670 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:02.888462 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:02.889211 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:02.888718 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:16:03.110599 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.110570 2577 generic.go:356] "Generic (PLEG): container finished" podID="74e989c5-7d7b-44b0-ba04-f5bc769838d2" containerID="4026c2941083ef606c86082d07d47343ac8d38747567745d0b1c37e58c6e0242" exitCode=0 Sep 19 04:16:03.110768 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.110653 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" event={"ID":"74e989c5-7d7b-44b0-ba04-f5bc769838d2","Type":"ContainerDied","Data":"4026c2941083ef606c86082d07d47343ac8d38747567745d0b1c37e58c6e0242"} Sep 19 04:16:03.114201 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.114183 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wkj2s_a151acea-4940-48b8-b206-571d1ea4fb4f/ovn-acl-logging/0.log" Sep 19 04:16:03.114630 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.114604 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerStarted","Data":"969ab74fb368135751e1cb1c85228bf36a4fa483617d3f625d891f72c875c427"} Sep 19 04:16:03.114842 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.114823 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:16:03.114930 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.114893 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:16:03.114995 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.114980 2577 scope.go:117] "RemoveContainer" containerID="b20c99dc7219fecd1e72e43e0aa9f54211c2bf53cac8250c9204258c732ce250" Sep 19 04:16:03.131365 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.131342 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:16:03.131461 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.131452 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:16:03.313607 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.313573 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:03.391304 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.391252 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:03.888255 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.888230 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:03.888418 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:03.888330 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:16:03.888418 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:03.888395 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:03.888534 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:03.888514 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:16:04.122019 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.121994 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wkj2s_a151acea-4940-48b8-b206-571d1ea4fb4f/ovn-acl-logging/0.log" Sep 19 04:16:04.122552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.122527 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" event={"ID":"a151acea-4940-48b8-b206-571d1ea4fb4f","Type":"ContainerStarted","Data":"cbb6121847401d5039520d257b69dc73d93862c42252dc983f5abc75dc9898e3"} Sep 19 04:16:04.122768 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.122753 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 19 04:16:04.160702 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.160616 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" podStartSLOduration=10.181029419 podStartE2EDuration="28.160599226s" totalImagesPullingTime="17.979569807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:04.16009446 +0000 UTC m=+30.481192605" watchObservedRunningTime="2026-09-19 04:16:04.160599226 +0000 UTC m=+30.481697404" Sep 19 04:16:04.319493 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.319324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8lnq4"] Sep 19 04:16:04.319650 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.319636 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:04.319779 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:04.319756 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:16:04.322993 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.322970 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nqvp6"] Sep 19 04:16:04.323123 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.323109 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:04.323269 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:04.323240 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:16:04.323454 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.323435 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mkdj2"] Sep 19 04:16:04.323529 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.323519 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:04.323613 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:04.323596 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:16:04.388124 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.388086 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:04.465645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:04.465566 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:05.126797 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:05.126760 2577 generic.go:356] "Generic (PLEG): container finished" podID="74e989c5-7d7b-44b0-ba04-f5bc769838d2" containerID="24646a220dc1d4754b248794d0d1aa541e5257a516123fdd2cd46c36a948c18e" exitCode=0 Sep 19 04:16:05.127287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:05.126845 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" event={"ID":"74e989c5-7d7b-44b0-ba04-f5bc769838d2","Type":"ContainerDied","Data":"24646a220dc1d4754b248794d0d1aa541e5257a516123fdd2cd46c36a948c18e"} Sep 19 04:16:05.127287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:05.126913 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 19 04:16:05.889475 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:05.889447 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:05.889633 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:05.889548 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:16:05.889633 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:05.889628 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:05.889744 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:05.889725 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:16:05.889804 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:05.889766 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:05.889857 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:05.889842 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:16:06.587855 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:06.587823 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:06.739550 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:06.739463 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:06.966199 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:06.966076 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:07.072199 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.072123 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:07.133713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.133680 2577 generic.go:356] "Generic (PLEG): container finished" podID="74e989c5-7d7b-44b0-ba04-f5bc769838d2" containerID="6a0175ec60a19ae0af1bb7f43c356a80d7f5f73ba44b8d5a13083a9d0ccf59cc" exitCode=0 Sep 19 04:16:07.133850 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.133722 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" event={"ID":"74e989c5-7d7b-44b0-ba04-f5bc769838d2","Type":"ContainerDied","Data":"6a0175ec60a19ae0af1bb7f43c356a80d7f5f73ba44b8d5a13083a9d0ccf59cc"} Sep 19 04:16:07.731089 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.731055 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:16:07.731711 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.731292 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 19 04:16:07.748581 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.748536 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" podUID="a151acea-4940-48b8-b206-571d1ea4fb4f" containerName="ovnkube-controller" probeResult="failure" output="" Sep 19 04:16:07.759532 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.759503 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" podUID="a151acea-4940-48b8-b206-571d1ea4fb4f" containerName="ovnkube-controller" probeResult="failure" output="" Sep 19 04:16:07.888639 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.888606 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:07.888794 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.888613 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:07.888794 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:07.888717 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:16:07.888794 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:07.888730 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:07.888971 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:07.888835 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:16:07.888971 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:07.888929 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:16:09.592074 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:09.592037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:09.592074 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:09.592079 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:09.592600 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:09.592102 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:09.592600 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.592217 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 04:16:09.592600 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.592222 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:16:09.592600 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.592238 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 04:16:09.592600 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.592254 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 04:16:09.592600 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.592267 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dd4bt for pod openshift-network-diagnostics/network-check-target-nqvp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:16:09.592600 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.592273 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret podName:4eea2ebc-5430-4f00-9ed7-492efe045fb0 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:41.592255855 +0000 UTC m=+67.913353992 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret") pod "global-pull-secret-syncer-8lnq4" (UID: "4eea2ebc-5430-4f00-9ed7-492efe045fb0") : object "kube-system"/"original-pull-secret" not registered Sep 19 04:16:09.592600 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.592292 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs podName:c9d8399c-1bc0-49da-8573-c6e428deebca nodeName:}" failed. No retries permitted until 2026-09-19 04:16:41.592282345 +0000 UTC m=+67.913380474 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs") pod "network-metrics-daemon-mkdj2" (UID: "c9d8399c-1bc0-49da-8573-c6e428deebca") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 04:16:09.592600 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.592305 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt podName:f184eae1-445c-4516-982f-908ad6ed1dda nodeName:}" failed. No retries permitted until 2026-09-19 04:16:41.592298286 +0000 UTC m=+67.913396433 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dd4bt" (UniqueName: "kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt") pod "network-check-target-nqvp6" (UID: "f184eae1-445c-4516-982f-908ad6ed1dda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 04:16:09.888271 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:09.888197 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:09.888444 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:09.888197 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:09.888444 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.888307 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lnq4" podUID="4eea2ebc-5430-4f00-9ed7-492efe045fb0" Sep 19 04:16:09.888444 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.888416 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdj2" podUID="c9d8399c-1bc0-49da-8573-c6e428deebca" Sep 19 04:16:09.888542 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:09.888458 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:09.888542 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:09.888532 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nqvp6" podUID="f184eae1-445c-4516-982f-908ad6ed1dda" Sep 19 04:16:10.074009 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.073973 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeReady" Sep 19 04:16:10.074244 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.074132 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 19 04:16:10.132630 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.132598 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xzvch"] Sep 19 04:16:10.149509 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.149433 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gcq92"] Sep 19 04:16:10.149672 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.149577 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.153796 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.153769 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 19 04:16:10.153937 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.153923 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-92kgd\"" Sep 19 04:16:10.154045 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.154029 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 19 04:16:10.163436 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.163414 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xzvch"] Sep 19 04:16:10.163555 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.163460 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gcq92"] Sep 19 04:16:10.163555 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.163516 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gcq92" Sep 19 04:16:10.165994 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.165975 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 19 04:16:10.166119 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.166063 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 19 04:16:10.166119 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.166092 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 19 04:16:10.166248 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.166144 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wzd8t\"" Sep 19 04:16:10.296801 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.296765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rklfh\" (UniqueName: \"kubernetes.io/projected/58784ca9-33f1-4d36-aefc-455a3568bf85-kube-api-access-rklfh\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.297003 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.296814 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4khl\" (UniqueName: \"kubernetes.io/projected/509b6215-7037-419b-8ca4-65d1c028e244-kube-api-access-b4khl\") pod \"ingress-canary-gcq92\" (UID: \"509b6215-7037-419b-8ca4-65d1c028e244\") " pod="openshift-ingress-canary/ingress-canary-gcq92" Sep 19 04:16:10.297003 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.296840 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.297003 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.296908 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58784ca9-33f1-4d36-aefc-455a3568bf85-tmp-dir\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.297003 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.296949 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/509b6215-7037-419b-8ca4-65d1c028e244-cert\") pod \"ingress-canary-gcq92\" (UID: \"509b6215-7037-419b-8ca4-65d1c028e244\") " pod="openshift-ingress-canary/ingress-canary-gcq92" Sep 19 04:16:10.297003 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.296984 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58784ca9-33f1-4d36-aefc-455a3568bf85-config-volume\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.311014 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.310980 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:10.383786 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.383754 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:10.397711 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.397670 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58784ca9-33f1-4d36-aefc-455a3568bf85-config-volume\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.397867 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.397729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rklfh\" (UniqueName: \"kubernetes.io/projected/58784ca9-33f1-4d36-aefc-455a3568bf85-kube-api-access-rklfh\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.397867 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.397762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4khl\" (UniqueName: \"kubernetes.io/projected/509b6215-7037-419b-8ca4-65d1c028e244-kube-api-access-b4khl\") pod \"ingress-canary-gcq92\" (UID: \"509b6215-7037-419b-8ca4-65d1c028e244\") " pod="openshift-ingress-canary/ingress-canary-gcq92" Sep 19 04:16:10.397867 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.397786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.397867 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.397825 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58784ca9-33f1-4d36-aefc-455a3568bf85-tmp-dir\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.397867 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.397862 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/509b6215-7037-419b-8ca4-65d1c028e244-cert\") pod \"ingress-canary-gcq92\" (UID: \"509b6215-7037-419b-8ca4-65d1c028e244\") " pod="openshift-ingress-canary/ingress-canary-gcq92" Sep 19 04:16:10.398113 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:10.397918 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 04:16:10.398113 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:10.397981 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls podName:58784ca9-33f1-4d36-aefc-455a3568bf85 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:10.897962258 +0000 UTC m=+37.219060383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls") pod "dns-default-xzvch" (UID: "58784ca9-33f1-4d36-aefc-455a3568bf85") : secret "dns-default-metrics-tls" not found Sep 19 04:16:10.398304 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.398281 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58784ca9-33f1-4d36-aefc-455a3568bf85-tmp-dir\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.398372 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.398353 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58784ca9-33f1-4d36-aefc-455a3568bf85-config-volume\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.402625 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.402438 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/509b6215-7037-419b-8ca4-65d1c028e244-cert\") pod \"ingress-canary-gcq92\" (UID: \"509b6215-7037-419b-8ca4-65d1c028e244\") " pod="openshift-ingress-canary/ingress-canary-gcq92" Sep 19 04:16:10.407830 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.407800 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4khl\" (UniqueName: \"kubernetes.io/projected/509b6215-7037-419b-8ca4-65d1c028e244-kube-api-access-b4khl\") pod \"ingress-canary-gcq92\" (UID: \"509b6215-7037-419b-8ca4-65d1c028e244\") " pod="openshift-ingress-canary/ingress-canary-gcq92" Sep 19 04:16:10.407970 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.407950 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rklfh\" (UniqueName: \"kubernetes.io/projected/58784ca9-33f1-4d36-aefc-455a3568bf85-kube-api-access-rklfh\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.473201 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.473143 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gcq92" Sep 19 04:16:10.839326 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.839289 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p5dtn"] Sep 19 04:16:10.865895 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.865862 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-9c66fb948-4h58s"] Sep 19 04:16:10.866066 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.866004 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:10.868847 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.868822 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 19 04:16:10.869017 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.868824 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 19 04:16:10.869707 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.869689 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-prc7h\"" Sep 19 04:16:10.869707 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.869704 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 19 04:16:10.869865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.869773 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 19 04:16:10.875903 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.875875 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 19 04:16:10.881988 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.881966 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p5dtn"] Sep 19 04:16:10.882077 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.882004 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9c66fb948-4h58s"] Sep 19 04:16:10.882115 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.882075 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:10.884769 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.884750 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jgbds\"" Sep 19 04:16:10.885255 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.885235 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 19 04:16:10.885355 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.885314 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 19 04:16:10.885561 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.885543 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 19 04:16:10.885643 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.885572 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 19 04:16:10.885705 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.885693 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 19 04:16:10.885949 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.885935 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 19 04:16:10.901622 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:10.901594 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:10.901780 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:10.901762 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 04:16:10.901846 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:10.901829 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls podName:58784ca9-33f1-4d36-aefc-455a3568bf85 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:11.901808944 +0000 UTC m=+38.222907082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls") pod "dns-default-xzvch" (UID: "58784ca9-33f1-4d36-aefc-455a3568bf85") : secret "dns-default-metrics-tls" not found Sep 19 04:16:11.002277 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002240 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv9xx\" (UniqueName: \"kubernetes.io/projected/133f6fcb-808b-4ae3-a06d-822efa9184a5-kube-api-access-rv9xx\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.002457 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002296 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-default-certificate\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.002457 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f534b12-a447-453d-aa10-9118ac1b6d53-tmp\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.002457 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f534b12-a447-453d-aa10-9118ac1b6d53-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.002608 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f534b12-a447-453d-aa10-9118ac1b6d53-serving-cert\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.002608 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002475 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpnn6\" (UniqueName: \"kubernetes.io/projected/0f534b12-a447-453d-aa10-9118ac1b6d53-kube-api-access-kpnn6\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.002608 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002502 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.002608 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002550 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-stats-auth\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.002608 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002589 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0f534b12-a447-453d-aa10-9118ac1b6d53-snapshots\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.002816 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f534b12-a447-453d-aa10-9118ac1b6d53-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.002816 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.002649 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.034554 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.034479 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:11.100755 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.100667 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:11.103621 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103593 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-default-certificate\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.103761 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103637 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f534b12-a447-453d-aa10-9118ac1b6d53-tmp\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.103761 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103662 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f534b12-a447-453d-aa10-9118ac1b6d53-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.103761 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103709 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f534b12-a447-453d-aa10-9118ac1b6d53-serving-cert\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.103761 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpnn6\" (UniqueName: \"kubernetes.io/projected/0f534b12-a447-453d-aa10-9118ac1b6d53-kube-api-access-kpnn6\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.103965 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103764 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.103965 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-stats-auth\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.103965 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103810 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0f534b12-a447-453d-aa10-9118ac1b6d53-snapshots\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.103965 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f534b12-a447-453d-aa10-9118ac1b6d53-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.103965 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103859 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.103965 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.103920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv9xx\" (UniqueName: \"kubernetes.io/projected/133f6fcb-808b-4ae3-a06d-822efa9184a5-kube-api-access-rv9xx\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.104249 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.104035 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f534b12-a447-453d-aa10-9118ac1b6d53-tmp\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.104249 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:11.104218 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:11.604195925 +0000 UTC m=+37.925294069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : configmap references non-existent config key: service-ca.crt Sep 19 04:16:11.104353 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:11.104298 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 04:16:11.104407 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:11.104358 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:11.604340096 +0000 UTC m=+37.925438225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : secret "router-metrics-certs-default" not found Sep 19 04:16:11.104536 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.104511 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f534b12-a447-453d-aa10-9118ac1b6d53-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.104610 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.104588 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0f534b12-a447-453d-aa10-9118ac1b6d53-snapshots\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.104662 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.104593 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f534b12-a447-453d-aa10-9118ac1b6d53-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.106825 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.106803 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-default-certificate\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.106915 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.106819 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-stats-auth\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.106967 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.106948 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f534b12-a447-453d-aa10-9118ac1b6d53-serving-cert\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.117070 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.117044 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv9xx\" (UniqueName: \"kubernetes.io/projected/133f6fcb-808b-4ae3-a06d-822efa9184a5-kube-api-access-rv9xx\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.118127 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.118110 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpnn6\" (UniqueName: \"kubernetes.io/projected/0f534b12-a447-453d-aa10-9118ac1b6d53-kube-api-access-kpnn6\") pod \"insights-operator-6f9c644dd9-p5dtn\" (UID: \"0f534b12-a447-453d-aa10-9118ac1b6d53\") " pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.177770 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.177737 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" Sep 19 04:16:11.608808 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.608771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.608808 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.608816 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:11.608985 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:11.608951 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 04:16:11.608985 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:11.608960 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:12.608941743 +0000 UTC m=+38.930039867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : configmap references non-existent config key: service-ca.crt Sep 19 04:16:11.609058 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:11.608998 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:12.608984742 +0000 UTC m=+38.930082866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : secret "router-metrics-certs-default" not found Sep 19 04:16:11.804280 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.804164 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gcq92"] Sep 19 04:16:11.806281 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.806258 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p5dtn"] Sep 19 04:16:11.858425 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:11.858380 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod509b6215_7037_419b_8ca4_65d1c028e244.slice/crio-ad7f5b7d5f2e832c118885af83a8affc766c0ba693405d8287ab509efafc5d55 WatchSource:0}: Error finding container ad7f5b7d5f2e832c118885af83a8affc766c0ba693405d8287ab509efafc5d55: Status 404 returned error can't find the container with id ad7f5b7d5f2e832c118885af83a8affc766c0ba693405d8287ab509efafc5d55 Sep 19 04:16:11.859042 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:11.859011 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f534b12_a447_453d_aa10_9118ac1b6d53.slice/crio-93b3163f51e4e378355e257545aea198f412c693f38acdd9a95938e27cd99c52 WatchSource:0}: Error finding container 93b3163f51e4e378355e257545aea198f412c693f38acdd9a95938e27cd99c52: Status 404 returned error can't find the container with id 93b3163f51e4e378355e257545aea198f412c693f38acdd9a95938e27cd99c52 Sep 19 04:16:11.888139 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.888114 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:11.888260 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.888166 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:11.888260 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.888172 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:11.891909 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.891891 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 19 04:16:11.892630 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.892613 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dswnr\"" Sep 19 04:16:11.892809 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.892796 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 19 04:16:11.893481 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.893463 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 19 04:16:11.893762 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.893740 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qntv9\"" Sep 19 04:16:11.896242 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.896226 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 19 04:16:11.910508 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:11.910490 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:11.910628 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:11.910614 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 04:16:11.910676 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:11.910667 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls podName:58784ca9-33f1-4d36-aefc-455a3568bf85 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:13.910650513 +0000 UTC m=+40.231748637 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls") pod "dns-default-xzvch" (UID: "58784ca9-33f1-4d36-aefc-455a3568bf85") : secret "dns-default-metrics-tls" not found Sep 19 04:16:12.150916 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:12.150885 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" event={"ID":"74e989c5-7d7b-44b0-ba04-f5bc769838d2","Type":"ContainerStarted","Data":"8f4c8e06a0b4efdd38a0015e557977d8175f71c7a0d0d2e0b5d07e9780480092"} Sep 19 04:16:12.152055 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:12.152026 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" event={"ID":"0f534b12-a447-453d-aa10-9118ac1b6d53","Type":"ContainerStarted","Data":"93b3163f51e4e378355e257545aea198f412c693f38acdd9a95938e27cd99c52"} Sep 19 04:16:12.153177 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:12.153140 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gcq92" event={"ID":"509b6215-7037-419b-8ca4-65d1c028e244","Type":"ContainerStarted","Data":"ad7f5b7d5f2e832c118885af83a8affc766c0ba693405d8287ab509efafc5d55"} Sep 19 04:16:12.616467 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:12.616426 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:12.616656 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:12.616488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:12.616722 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:12.616692 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 04:16:12.616776 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:12.616757 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:14.616735468 +0000 UTC m=+40.937833595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : secret "router-metrics-certs-default" not found Sep 19 04:16:12.617114 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:12.617099 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:14.617087853 +0000 UTC m=+40.938185977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : configmap references non-existent config key: service-ca.crt Sep 19 04:16:13.159466 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:13.159227 2577 generic.go:356] "Generic (PLEG): container finished" podID="74e989c5-7d7b-44b0-ba04-f5bc769838d2" containerID="8f4c8e06a0b4efdd38a0015e557977d8175f71c7a0d0d2e0b5d07e9780480092" exitCode=0 Sep 19 04:16:13.159933 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:13.159266 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" event={"ID":"74e989c5-7d7b-44b0-ba04-f5bc769838d2","Type":"ContainerDied","Data":"8f4c8e06a0b4efdd38a0015e557977d8175f71c7a0d0d2e0b5d07e9780480092"} Sep 19 04:16:13.927497 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:13.927458 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:13.927705 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:13.927590 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 04:16:13.927705 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:13.927658 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls podName:58784ca9-33f1-4d36-aefc-455a3568bf85 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:17.92763953 +0000 UTC m=+44.248737677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls") pod "dns-default-xzvch" (UID: "58784ca9-33f1-4d36-aefc-455a3568bf85") : secret "dns-default-metrics-tls" not found Sep 19 04:16:14.166328 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:14.166246 2577 generic.go:356] "Generic (PLEG): container finished" podID="74e989c5-7d7b-44b0-ba04-f5bc769838d2" containerID="7288d4ffe3fc58b69201522381de1d5cf382a066f8f58cdfa70eff1aa043f3d1" exitCode=0 Sep 19 04:16:14.166328 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:14.166291 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" event={"ID":"74e989c5-7d7b-44b0-ba04-f5bc769838d2","Type":"ContainerDied","Data":"7288d4ffe3fc58b69201522381de1d5cf382a066f8f58cdfa70eff1aa043f3d1"} Sep 19 04:16:14.633758 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:14.633728 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:14.633758 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:14.633764 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:14.634017 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:14.633894 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 04:16:14.634017 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:14.633912 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:18.633890562 +0000 UTC m=+44.954988701 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : configmap references non-existent config key: service-ca.crt Sep 19 04:16:14.634017 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:14.633940 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:18.633930136 +0000 UTC m=+44.955028266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : secret "router-metrics-certs-default" not found Sep 19 04:16:15.101063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.101028 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:15.172601 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.172523 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" event={"ID":"74e989c5-7d7b-44b0-ba04-f5bc769838d2","Type":"ContainerStarted","Data":"ebe820a1117f30ff14a9ceb9416e97a2192f73c66d5494f346a866d3cc162180"} Sep 19 04:16:15.174699 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.174668 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" event={"ID":"0f534b12-a447-453d-aa10-9118ac1b6d53","Type":"ContainerStarted","Data":"4ae7bcde029003d978721036fade5e4649002324c5938d0728f59e20f4dce769"} Sep 19 04:16:15.176189 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.176168 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gcq92" event={"ID":"509b6215-7037-419b-8ca4-65d1c028e244","Type":"ContainerStarted","Data":"c6ba95e22424240369187512d11ff6a879274df51b7d9789c34a0b3ce133fbff"} Sep 19 04:16:15.185162 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.185134 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:15.223054 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.223006 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fr7lx" podStartSLOduration=9.78268974 podStartE2EDuration="39.222988857s" totalImagesPullingTime="29.440299117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:15.219366715 +0000 UTC m=+41.540464863" watchObservedRunningTime="2026-09-19 04:16:15.222988857 +0000 UTC m=+41.544087004" Sep 19 04:16:15.278727 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.278676 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gcq92" podStartSLOduration=2.279767096 podStartE2EDuration="5.27865586s" totalImagesPullingTime="2.998888764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:15.252393855 +0000 UTC m=+41.573492010" watchObservedRunningTime="2026-09-19 04:16:15.27865586 +0000 UTC m=+41.599753997" Sep 19 04:16:15.432716 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.432627 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:15.512543 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.512506 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:15.964191 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.964126 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:15.979238 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.979180 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" podStartSLOduration=2.985793263 podStartE2EDuration="5.979164206s" totalImagesPullingTime="2.993370943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:15.279312918 +0000 UTC m=+41.600411064" watchObservedRunningTime="2026-09-19 04:16:15.979164206 +0000 UTC m=+42.300262344" Sep 19 04:16:15.979974 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.979947 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bm85n"] Sep 19 04:16:15.982200 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.982185 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:15.985040 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.985010 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 19 04:16:15.985851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.985831 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 19 04:16:15.986265 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:15.986250 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fgpsd\"" Sep 19 04:16:16.007573 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.007548 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bm85n"] Sep 19 04:16:16.145666 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.145615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.145865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.145691 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h972c\" (UniqueName: \"kubernetes.io/projected/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-api-access-h972c\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.145865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.145736 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-crio-socket\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.145865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.145778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-data-volume\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.145865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.145799 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.246859 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.246765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h972c\" (UniqueName: \"kubernetes.io/projected/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-api-access-h972c\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.246859 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.246820 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-crio-socket\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.246859 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.246858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-data-volume\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.247378 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.246876 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.247378 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:16.246953 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 04:16:16.247378 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.246988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.247378 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:16.247000 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:16.746987876 +0000 UTC m=+43.068086000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : configmap "kube-rbac-proxy" not found Sep 19 04:16:16.247378 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.247083 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-crio-socket\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.247378 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:16.247142 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 04:16:16.247378 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:16.247228 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:16.747209847 +0000 UTC m=+43.068307982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : secret "insights-runtime-extractor-tls" not found Sep 19 04:16:16.247378 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.247276 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-data-volume\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.260772 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.260752 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h972c\" (UniqueName: \"kubernetes.io/projected/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-api-access-h972c\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.751246 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.751199 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.751425 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:16.751297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:16.751425 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:16.751317 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 04:16:16.751425 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:16.751375 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:17.751359583 +0000 UTC m=+44.072457707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : configmap "kube-rbac-proxy" not found Sep 19 04:16:16.751524 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:16.751434 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 04:16:16.751524 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:16.751481 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:17.751469664 +0000 UTC m=+44.072567788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : secret "insights-runtime-extractor-tls" not found Sep 19 04:16:17.550115 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:17.550082 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:17.667576 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:17.667532 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:17.759518 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:17.759482 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:17.759692 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:17.759583 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:17.759692 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:17.759636 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 04:16:17.759781 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:17.759692 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 04:16:17.759781 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:17.759702 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:19.759687093 +0000 UTC m=+46.080785221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : secret "insights-runtime-extractor-tls" not found Sep 19 04:16:17.759781 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:17.759736 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:19.759723882 +0000 UTC m=+46.080822006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : configmap "kube-rbac-proxy" not found Sep 19 04:16:17.875621 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:17.875548 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rjjrv_ac630975-bfc0-46ba-9195-78394cc767ea/dns-node-resolver/0.log" Sep 19 04:16:17.961646 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:17.961601 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:17.961822 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:17.961720 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 04:16:17.961822 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:17.961772 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls podName:58784ca9-33f1-4d36-aefc-455a3568bf85 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:25.961759217 +0000 UTC m=+52.282857341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls") pod "dns-default-xzvch" (UID: "58784ca9-33f1-4d36-aefc-455a3568bf85") : secret "dns-default-metrics-tls" not found Sep 19 04:16:18.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:18.668814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:18.668851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:18.668851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:18.669422 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:18.669005 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 04:16:18.669422 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:18.669013 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:26.668988047 +0000 UTC m=+52.990086173 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : configmap references non-existent config key: service-ca.crt Sep 19 04:16:18.669422 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:18.669042 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:26.669031246 +0000 UTC m=+52.990129369 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : secret "router-metrics-certs-default" not found Sep 19 04:16:18.677067 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:18.677048 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-645z6_c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd/node-ca/0.log" Sep 19 04:16:19.277447 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:19.277414 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gcq92_509b6215-7037-419b-8ca4-65d1c028e244/serve-healthcheck-canary/0.log" Sep 19 04:16:19.776531 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:19.776498 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:19.776913 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:19.776580 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:19.776913 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:19.776633 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 04:16:19.776913 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:19.776694 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 04:16:19.776913 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:19.776716 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:23.776696587 +0000 UTC m=+50.097794730 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : configmap "kube-rbac-proxy" not found Sep 19 04:16:19.776913 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:19.776733 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:23.776722619 +0000 UTC m=+50.097820744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : secret "insights-runtime-extractor-tls" not found Sep 19 04:16:22.628544 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.628509 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6"] Sep 19 04:16:22.631165 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.631140 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6" Sep 19 04:16:22.635965 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.635940 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 19 04:16:22.636067 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.635975 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 19 04:16:22.636067 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.636002 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-fqgzp\"" Sep 19 04:16:22.650735 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.650706 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6"] Sep 19 04:16:22.800978 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.800937 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fntt5\" (UniqueName: \"kubernetes.io/projected/b969e868-5636-4a8c-af91-d8520bb37efc-kube-api-access-fntt5\") pod \"volume-data-source-validator-5c4db8bdf9-7dvk6\" (UID: \"b969e868-5636-4a8c-af91-d8520bb37efc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6" Sep 19 04:16:22.901951 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.901831 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fntt5\" (UniqueName: \"kubernetes.io/projected/b969e868-5636-4a8c-af91-d8520bb37efc-kube-api-access-fntt5\") pod \"volume-data-source-validator-5c4db8bdf9-7dvk6\" (UID: \"b969e868-5636-4a8c-af91-d8520bb37efc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6" Sep 19 04:16:22.917917 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.917888 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fntt5\" (UniqueName: \"kubernetes.io/projected/b969e868-5636-4a8c-af91-d8520bb37efc-kube-api-access-fntt5\") pod \"volume-data-source-validator-5c4db8bdf9-7dvk6\" (UID: \"b969e868-5636-4a8c-af91-d8520bb37efc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6" Sep 19 04:16:22.939531 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:22.939508 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6" Sep 19 04:16:23.216244 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:23.216075 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6"] Sep 19 04:16:23.218573 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:23.218543 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb969e868_5636_4a8c_af91_d8520bb37efc.slice/crio-5e50896e8c2c050253b8b7346dc53a4952faf5697e85c0b9591b69c9dda56b9c WatchSource:0}: Error finding container 5e50896e8c2c050253b8b7346dc53a4952faf5697e85c0b9591b69c9dda56b9c: Status 404 returned error can't find the container with id 5e50896e8c2c050253b8b7346dc53a4952faf5697e85c0b9591b69c9dda56b9c Sep 19 04:16:23.810391 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:23.810355 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:23.810854 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:23.810457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:23.810854 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:23.810494 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 04:16:23.810854 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:23.810592 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 04:16:23.810854 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:23.810602 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:31.810552602 +0000 UTC m=+58.131650726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : configmap "kube-rbac-proxy" not found Sep 19 04:16:23.810854 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:23.810643 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:31.810627409 +0000 UTC m=+58.131725548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : secret "insights-runtime-extractor-tls" not found Sep 19 04:16:24.200788 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:24.200702 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6" event={"ID":"b969e868-5636-4a8c-af91-d8520bb37efc","Type":"ContainerStarted","Data":"5e50896e8c2c050253b8b7346dc53a4952faf5697e85c0b9591b69c9dda56b9c"} Sep 19 04:16:25.204285 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:25.204248 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6" event={"ID":"b969e868-5636-4a8c-af91-d8520bb37efc","Type":"ContainerStarted","Data":"f0aa0718e56a6ef825c481b61ebe8891c84f058a19b4bd3e6d2afc685de050c4"} Sep 19 04:16:25.223300 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:25.223255 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-7dvk6" podStartSLOduration=2.177980172 podStartE2EDuration="3.223239693s" totalImagesPullingTime="1.045259521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:16:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:25.222395531 +0000 UTC m=+51.543493678" watchObservedRunningTime="2026-09-19 04:16:25.223239693 +0000 UTC m=+51.544337838" Sep 19 04:16:25.425081 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:25.425043 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:25.516689 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:25.516607 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:26.028117 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:26.028081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:26.028264 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:26.028221 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 04:16:26.028314 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:26.028286 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls podName:58784ca9-33f1-4d36-aefc-455a3568bf85 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:42.028272586 +0000 UTC m=+68.349370709 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls") pod "dns-default-xzvch" (UID: "58784ca9-33f1-4d36-aefc-455a3568bf85") : secret "dns-default-metrics-tls" not found Sep 19 04:16:26.733640 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:26.733600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:26.733640 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:26.733647 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:26.734070 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:26.733802 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 04:16:26.734070 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:26.733802 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:42.733783972 +0000 UTC m=+69.054882114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : configmap references non-existent config key: service-ca.crt Sep 19 04:16:26.734070 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:26.733860 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs podName:133f6fcb-808b-4ae3-a06d-822efa9184a5 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:42.733844674 +0000 UTC m=+69.054942804 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs") pod "router-default-9c66fb948-4h58s" (UID: "133f6fcb-808b-4ae3-a06d-822efa9184a5") : secret "router-metrics-certs-default" not found Sep 19 04:16:28.470906 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:28.470869 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:28.591354 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:28.591314 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:28.702831 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:28.702795 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:28.781340 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:28.781303 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:28.882449 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:28.882408 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:28.978127 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:28.978093 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:29.100649 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:29.100560 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:29.310659 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:29.310621 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:29.407194 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:29.407077 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:30.534889 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:30.534846 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:30.949767 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:30.949681 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:31.048449 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:31.048416 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:31.872764 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:31.872725 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:31.873148 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:31.872818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:31.873148 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:31.872982 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 04:16:31.873148 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:31.873057 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls podName:c2a4613e-bd9e-4c5b-96e6-224b6615fad6 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:47.873039884 +0000 UTC m=+74.194138008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bm85n" (UID: "c2a4613e-bd9e-4c5b-96e6-224b6615fad6") : secret "insights-runtime-extractor-tls" not found Sep 19 04:16:31.873363 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:31.873344 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:32.135677 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:32.135585 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:33.280780 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:33.280742 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:33.384098 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:33.384065 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:33.595811 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:33.595714 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:33.750300 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:33.750256 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:34.096301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:34.096270 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:34.981076 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:34.981038 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:35.165472 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:35.165439 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:35.255836 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:35.255748 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:35.381638 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:35.381600 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:35.596579 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:35.596532 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:35.778397 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:35.778362 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:35.888179 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:35.888086 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:36.193272 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:36.193147 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:36.781391 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:36.781349 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:36.998819 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:36.998780 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:37.133246 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:37.133141 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:37.426291 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:37.426200 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:37.759119 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:37.759046 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wkj2s" Sep 19 04:16:37.897006 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:37.896975 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:38.003504 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:38.003464 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:38.176401 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:38.176361 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:38.811749 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:38.811709 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:39.087394 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:39.087296 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:39.458339 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:39.458260 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:39.605560 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:39.605527 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:39.810460 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:39.810426 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:40.217231 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:40.217123 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:40.783066 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:40.783022 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:41.365860 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.365821 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:41.576852 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.576809 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:41.639139 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.639050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:41.639139 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.639103 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:41.639139 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.639128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:41.641959 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.641941 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 19 04:16:41.642694 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.642676 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 19 04:16:41.642743 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.642677 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 19 04:16:41.652071 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.652043 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4eea2ebc-5430-4f00-9ed7-492efe045fb0-original-pull-secret\") pod \"global-pull-secret-syncer-8lnq4\" (UID: \"4eea2ebc-5430-4f00-9ed7-492efe045fb0\") " pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:41.652191 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.652094 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8399c-1bc0-49da-8573-c6e428deebca-metrics-certs\") pod \"network-metrics-daemon-mkdj2\" (UID: \"c9d8399c-1bc0-49da-8573-c6e428deebca\") " pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:41.652850 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.652835 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 19 04:16:41.662316 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.662294 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd4bt\" (UniqueName: \"kubernetes.io/projected/f184eae1-445c-4516-982f-908ad6ed1dda-kube-api-access-dd4bt\") pod \"network-check-target-nqvp6\" (UID: \"f184eae1-445c-4516-982f-908ad6ed1dda\") " pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:41.765565 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.765530 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:41.901680 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.901603 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qntv9\"" Sep 19 04:16:41.907448 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.907421 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dswnr\"" Sep 19 04:16:41.909524 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.909508 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:41.909610 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.909596 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lnq4" Sep 19 04:16:41.915923 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:41.915908 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdj2" Sep 19 04:16:42.042647 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.042608 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:42.044986 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.044958 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58784ca9-33f1-4d36-aefc-455a3568bf85-metrics-tls\") pod \"dns-default-xzvch\" (UID: \"58784ca9-33f1-4d36-aefc-455a3568bf85\") " pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:42.172832 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.172544 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:42.201955 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.201840 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8lnq4"] Sep 19 04:16:42.204514 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:42.204474 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eea2ebc_5430_4f00_9ed7_492efe045fb0.slice/crio-5535f04a3a4af3d2f7b832738efcc3e5b37af8570331cbe0b41010a6a6845acb WatchSource:0}: Error finding container 5535f04a3a4af3d2f7b832738efcc3e5b37af8570331cbe0b41010a6a6845acb: Status 404 returned error can't find the container with id 5535f04a3a4af3d2f7b832738efcc3e5b37af8570331cbe0b41010a6a6845acb Sep 19 04:16:42.206955 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.206931 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nqvp6"] Sep 19 04:16:42.207286 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.207265 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mkdj2"] Sep 19 04:16:42.208655 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:42.208632 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d8399c_1bc0_49da_8573_c6e428deebca.slice/crio-bc110c69236703269559fa34560302db795cb541d7fff4d853c021f9cf84465a WatchSource:0}: Error finding container bc110c69236703269559fa34560302db795cb541d7fff4d853c021f9cf84465a: Status 404 returned error can't find the container with id bc110c69236703269559fa34560302db795cb541d7fff4d853c021f9cf84465a Sep 19 04:16:42.210851 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:42.210827 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf184eae1_445c_4516_982f_908ad6ed1dda.slice/crio-68b1a4cb6b82ec9c96f4eac5d4aaebc9ef961df2bf0dfacb5d9ae08fed3a6b2c WatchSource:0}: Error finding container 68b1a4cb6b82ec9c96f4eac5d4aaebc9ef961df2bf0dfacb5d9ae08fed3a6b2c: Status 404 returned error can't find the container with id 68b1a4cb6b82ec9c96f4eac5d4aaebc9ef961df2bf0dfacb5d9ae08fed3a6b2c Sep 19 04:16:42.247440 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.247401 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkdj2" event={"ID":"c9d8399c-1bc0-49da-8573-c6e428deebca","Type":"ContainerStarted","Data":"bc110c69236703269559fa34560302db795cb541d7fff4d853c021f9cf84465a"} Sep 19 04:16:42.248600 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.248569 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nqvp6" event={"ID":"f184eae1-445c-4516-982f-908ad6ed1dda","Type":"ContainerStarted","Data":"68b1a4cb6b82ec9c96f4eac5d4aaebc9ef961df2bf0dfacb5d9ae08fed3a6b2c"} Sep 19 04:16:42.249637 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.249615 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8lnq4" event={"ID":"4eea2ebc-5430-4f00-9ed7-492efe045fb0","Type":"ContainerStarted","Data":"5535f04a3a4af3d2f7b832738efcc3e5b37af8570331cbe0b41010a6a6845acb"} Sep 19 04:16:42.263535 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.263513 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-92kgd\"" Sep 19 04:16:42.271904 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.271888 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:42.552592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.552559 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xzvch"] Sep 19 04:16:42.556938 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:42.556904 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58784ca9_33f1_4d36_aefc_455a3568bf85.slice/crio-23aad11221dee28b691dfac1e8c1a42ff768f06d99962fabb5f50353c2c99144 WatchSource:0}: Error finding container 23aad11221dee28b691dfac1e8c1a42ff768f06d99962fabb5f50353c2c99144: Status 404 returned error can't find the container with id 23aad11221dee28b691dfac1e8c1a42ff768f06d99962fabb5f50353c2c99144 Sep 19 04:16:42.750127 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.750087 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:42.750310 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.750139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:42.750896 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.750861 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133f6fcb-808b-4ae3-a06d-822efa9184a5-service-ca-bundle\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:42.760627 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.760597 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f6fcb-808b-4ae3-a06d-822efa9184a5-metrics-certs\") pod \"router-default-9c66fb948-4h58s\" (UID: \"133f6fcb-808b-4ae3-a06d-822efa9184a5\") " pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:42.791232 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:42.791123 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:43.006056 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:43.005766 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jgbds\"" Sep 19 04:16:43.012552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:43.012315 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:43.177918 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:43.177879 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:43.254825 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:43.254750 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xzvch" event={"ID":"58784ca9-33f1-4d36-aefc-455a3568bf85","Type":"ContainerStarted","Data":"23aad11221dee28b691dfac1e8c1a42ff768f06d99962fabb5f50353c2c99144"} Sep 19 04:16:43.616475 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:43.616389 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:43.640516 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:43.640134 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9c66fb948-4h58s"] Sep 19 04:16:43.643312 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:43.643282 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133f6fcb_808b_4ae3_a06d_822efa9184a5.slice/crio-0d461bfeb12a299d20e26c399dd97bd23d134670bd88f2fe22854db01f1dd5fc WatchSource:0}: Error finding container 0d461bfeb12a299d20e26c399dd97bd23d134670bd88f2fe22854db01f1dd5fc: Status 404 returned error can't find the container with id 0d461bfeb12a299d20e26c399dd97bd23d134670bd88f2fe22854db01f1dd5fc Sep 19 04:16:43.766516 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:43.766480 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:43.980269 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:43.980177 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:44.224477 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:44.224330 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:44.263144 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:44.263055 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9c66fb948-4h58s" event={"ID":"133f6fcb-808b-4ae3-a06d-822efa9184a5","Type":"ContainerStarted","Data":"ea3a50174b2d6dd0a380186041de4ba22fe177dd36672429c38fa02b0013801c"} Sep 19 04:16:44.263144 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:44.263132 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9c66fb948-4h58s" event={"ID":"133f6fcb-808b-4ae3-a06d-822efa9184a5","Type":"ContainerStarted","Data":"0d461bfeb12a299d20e26c399dd97bd23d134670bd88f2fe22854db01f1dd5fc"} Sep 19 04:16:44.266076 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:44.265641 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkdj2" event={"ID":"c9d8399c-1bc0-49da-8573-c6e428deebca","Type":"ContainerStarted","Data":"1bd20b24ae83f7cff31b0339c89ef1ceefbde55b506ba6f0da73e202216b7083"} Sep 19 04:16:44.266076 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:44.265692 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkdj2" event={"ID":"c9d8399c-1bc0-49da-8573-c6e428deebca","Type":"ContainerStarted","Data":"3f245e6d1a3250f8ece9d52d293983671a1af3f423907933c055514e157021b3"} Sep 19 04:16:44.318934 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:44.318867 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mkdj2" podStartSLOduration=67.189109069 podStartE2EDuration="1m8.318845049s" totalImagesPullingTime="1.12973598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:44.318509251 +0000 UTC m=+70.639607437" watchObservedRunningTime="2026-09-19 04:16:44.318845049 +0000 UTC m=+70.639943199" Sep 19 04:16:44.319809 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:44.319762 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-9c66fb948-4h58s" podStartSLOduration=34.319749209 podStartE2EDuration="34.319749209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:16:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:44.294763085 +0000 UTC m=+70.615861240" watchObservedRunningTime="2026-09-19 04:16:44.319749209 +0000 UTC m=+70.640847356" Sep 19 04:16:45.013053 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.012864 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:45.015976 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.015936 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:45.124042 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.123939 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:45.268077 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.267959 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:45.270222 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.270193 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:45.271722 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.271700 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-9c66fb948-4h58s" Sep 19 04:16:45.371551 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.371514 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:45.488170 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.488119 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:45.587986 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.587954 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:45.689653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.689622 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:45.781184 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.781082 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:45.987827 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:45.987729 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:46.076604 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:46.076567 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:47.190130 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.190049 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:47.278758 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.278719 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xzvch" event={"ID":"58784ca9-33f1-4d36-aefc-455a3568bf85","Type":"ContainerStarted","Data":"59de1cc9d7d3d3934f685a2f3caecc8de451f435b87fb4c1aa1de4f5b1f2a436"} Sep 19 04:16:47.279026 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.278806 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xzvch" event={"ID":"58784ca9-33f1-4d36-aefc-455a3568bf85","Type":"ContainerStarted","Data":"3f1deaffc49cb26b1ca9b9e3d0d62bdf6dcf1889ac6e97a16b9d0cedc7504b75"} Sep 19 04:16:47.279026 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.278859 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:47.280435 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.280405 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nqvp6" event={"ID":"f184eae1-445c-4516-982f-908ad6ed1dda","Type":"ContainerStarted","Data":"f9b1b83d9c33cfaed62827614aba785a85ce8580a9f3ca72d3e8aa113583a186"} Sep 19 04:16:47.280570 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.280481 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:16:47.281994 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.281969 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8lnq4" event={"ID":"4eea2ebc-5430-4f00-9ed7-492efe045fb0","Type":"ContainerStarted","Data":"cd558003a3a4166fa86b98b856a780f4b8e916e287cd00ec2a548a2b0b4f1afc"} Sep 19 04:16:47.301617 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.301528 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xzvch" podStartSLOduration=32.816213372 podStartE2EDuration="37.30151016s" totalImagesPullingTime="4.485296788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:47.300982164 +0000 UTC m=+73.622080311" watchObservedRunningTime="2026-09-19 04:16:47.30151016 +0000 UTC m=+73.622608307" Sep 19 04:16:47.333023 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.332825 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8lnq4" podStartSLOduration=66.483269991 podStartE2EDuration="1m11.332811034s" totalImagesPullingTime="4.849541043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:47.332293749 +0000 UTC m=+73.653391889" watchObservedRunningTime="2026-09-19 04:16:47.332811034 +0000 UTC m=+73.653909179" Sep 19 04:16:47.353868 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.353809 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nqvp6" podStartSLOduration=67.513429974 podStartE2EDuration="1m12.35379095s" totalImagesPullingTime="4.840360976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:47.352940556 +0000 UTC m=+73.674038702" watchObservedRunningTime="2026-09-19 04:16:47.35379095 +0000 UTC m=+73.674889117" Sep 19 04:16:47.388300 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.388256 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:47.477534 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.477501 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:47.895379 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.895351 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:47.897729 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:47.897708 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c2a4613e-bd9e-4c5b-96e6-224b6615fad6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bm85n\" (UID: \"c2a4613e-bd9e-4c5b-96e6-224b6615fad6\") " pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:48.093593 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:48.093560 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fgpsd\"" Sep 19 04:16:48.101251 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:48.101223 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bm85n" Sep 19 04:16:48.377792 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:48.377761 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bm85n"] Sep 19 04:16:48.380516 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:48.380485 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a4613e_bd9e_4c5b_96e6_224b6615fad6.slice/crio-9072fcb0799cf09b90ce8601dfff8b3090fbf997e847a3a07e83414d5ca40303 WatchSource:0}: Error finding container 9072fcb0799cf09b90ce8601dfff8b3090fbf997e847a3a07e83414d5ca40303: Status 404 returned error can't find the container with id 9072fcb0799cf09b90ce8601dfff8b3090fbf997e847a3a07e83414d5ca40303 Sep 19 04:16:48.599657 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:48.599572 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:48.704368 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:48.704283 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:49.295901 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:49.295819 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bm85n" event={"ID":"c2a4613e-bd9e-4c5b-96e6-224b6615fad6","Type":"ContainerStarted","Data":"d4200de43057f2d4cf813ea4b08ceedd8ce4a1c06f53b33fb1fe21aa9f83ea47"} Sep 19 04:16:49.295901 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:49.295896 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bm85n" event={"ID":"c2a4613e-bd9e-4c5b-96e6-224b6615fad6","Type":"ContainerStarted","Data":"df9ef095e88fd1ec278cec82c5dcede08dff2b35c5acad189ac29ac623567bc0"} Sep 19 04:16:49.296072 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:49.295917 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bm85n" event={"ID":"c2a4613e-bd9e-4c5b-96e6-224b6615fad6","Type":"ContainerStarted","Data":"9072fcb0799cf09b90ce8601dfff8b3090fbf997e847a3a07e83414d5ca40303"} Sep 19 04:16:50.145753 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:50.145714 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:50.246338 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:50.246302 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:50.301241 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:50.301184 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bm85n" event={"ID":"c2a4613e-bd9e-4c5b-96e6-224b6615fad6","Type":"ContainerStarted","Data":"8997f52eac30129aeb6e7b51a050796532693485f2e7dea029cc1dee238ecbba"} Sep 19 04:16:50.323778 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:50.323567 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bm85n" podStartSLOduration=33.598298056 podStartE2EDuration="35.32354484s" totalImagesPullingTime="1.725246784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:16:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:50.321963876 +0000 UTC m=+76.643062022" watchObservedRunningTime="2026-09-19 04:16:50.32354484 +0000 UTC m=+76.644642986" Sep 19 04:16:54.609492 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:54.609457 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:56.723659 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.723620 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn"] Sep 19 04:16:56.725710 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.725693 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.730090 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.730069 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 19 04:16:56.730275 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.730128 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 19 04:16:56.730982 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.730966 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 19 04:16:56.732180 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.732142 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 19 04:16:56.750535 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.750497 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn"] Sep 19 04:16:56.754453 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.754423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxs29\" (UniqueName: \"kubernetes.io/projected/13ec8341-725a-455e-9039-09136df97735-kube-api-access-xxs29\") pod \"klusterlet-addon-workmgr-c6d9d8c86-dm5pn\" (UID: \"13ec8341-725a-455e-9039-09136df97735\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.754584 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.754470 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13ec8341-725a-455e-9039-09136df97735-tmp\") pod \"klusterlet-addon-workmgr-c6d9d8c86-dm5pn\" (UID: \"13ec8341-725a-455e-9039-09136df97735\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.754584 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.754534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/13ec8341-725a-455e-9039-09136df97735-klusterlet-config\") pod \"klusterlet-addon-workmgr-c6d9d8c86-dm5pn\" (UID: \"13ec8341-725a-455e-9039-09136df97735\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.822993 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.822963 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-2j44x"] Sep 19 04:16:56.825066 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.825051 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" Sep 19 04:16:56.829681 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.829656 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fr6hb\"" Sep 19 04:16:56.830368 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.830348 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 19 04:16:56.831855 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.831827 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7564fc9dff-c7lgv"] Sep 19 04:16:56.833711 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.833691 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b7dfb47d4-246qx"] Sep 19 04:16:56.833816 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.833802 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:56.834363 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.834344 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 19 04:16:56.835569 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.835555 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:56.836609 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.836585 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"registry-dockercfg-82l8w\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-82l8w\"" type="*v1.Secret" Sep 19 04:16:56.838009 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.837988 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"installation-pull-secrets\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" type="*v1.Secret" Sep 19 04:16:56.838124 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.838005 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"image-registry-tls\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" type="*v1.Secret" Sep 19 04:16:56.838124 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.838018 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"image-registry-private-configuration\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" type="*v1.Secret" Sep 19 04:16:56.838124 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.838018 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Sep 19 04:16:56.838331 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.838133 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"service-ca\"" type="*v1.ConfigMap" Sep 19 04:16:56.838935 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.838916 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" type="*v1.ConfigMap" Sep 19 04:16:56.841561 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.841537 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Sep 19 04:16:56.841737 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.841718 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"oauth-serving-cert\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" type="*v1.ConfigMap" Sep 19 04:16:56.843392 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.843374 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"console-dockercfg-9h7ml\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-dockercfg-9h7ml\"" type="*v1.Secret" Sep 19 04:16:56.843561 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.843539 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"console-oauth-config\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-oauth-config\"" type="*v1.Secret" Sep 19 04:16:56.843603 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.843561 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"console-serving-cert\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-serving-cert\"" type="*v1.Secret" Sep 19 04:16:56.843603 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.843538 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"console-config\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-config\"" type="*v1.ConfigMap" Sep 19 04:16:56.853310 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.853287 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-2j44x"] Sep 19 04:16:56.855464 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.855445 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxs29\" (UniqueName: \"kubernetes.io/projected/13ec8341-725a-455e-9039-09136df97735-kube-api-access-xxs29\") pod \"klusterlet-addon-workmgr-c6d9d8c86-dm5pn\" (UID: \"13ec8341-725a-455e-9039-09136df97735\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.855557 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.855478 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13ec8341-725a-455e-9039-09136df97735-tmp\") pod \"klusterlet-addon-workmgr-c6d9d8c86-dm5pn\" (UID: \"13ec8341-725a-455e-9039-09136df97735\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.855557 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.855512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/13ec8341-725a-455e-9039-09136df97735-klusterlet-config\") pod \"klusterlet-addon-workmgr-c6d9d8c86-dm5pn\" (UID: \"13ec8341-725a-455e-9039-09136df97735\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.855651 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.855572 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/db9e87e7-0480-4019-9416-827502badf63-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-2j44x\" (UID: \"db9e87e7-0480-4019-9416-827502badf63\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" Sep 19 04:16:56.855651 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.855615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db9e87e7-0480-4019-9416-827502badf63-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-2j44x\" (UID: \"db9e87e7-0480-4019-9416-827502badf63\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" Sep 19 04:16:56.855839 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.855818 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13ec8341-725a-455e-9039-09136df97735-tmp\") pod \"klusterlet-addon-workmgr-c6d9d8c86-dm5pn\" (UID: \"13ec8341-725a-455e-9039-09136df97735\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.858330 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.858310 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/13ec8341-725a-455e-9039-09136df97735-klusterlet-config\") pod \"klusterlet-addon-workmgr-c6d9d8c86-dm5pn\" (UID: \"13ec8341-725a-455e-9039-09136df97735\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.860121 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.860100 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7dfb47d4-246qx"] Sep 19 04:16:56.861093 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.861065 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7564fc9dff-c7lgv"] Sep 19 04:16:56.873245 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.873223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxs29\" (UniqueName: \"kubernetes.io/projected/13ec8341-725a-455e-9039-09136df97735-kube-api-access-xxs29\") pod \"klusterlet-addon-workmgr-c6d9d8c86-dm5pn\" (UID: \"13ec8341-725a-455e-9039-09136df97735\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:56.918996 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.918966 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7564fc9dff-c7lgv"] Sep 19 04:16:56.919191 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:56.919171 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-jkkpj registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-jkkpj registry-certificates registry-tls trusted-ca]: context canceled" pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" podUID="28d43e91-51bd-44e2-91c9-f5425a8eae54" Sep 19 04:16:56.956557 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956521 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-service-ca\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:56.956633 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956563 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28d43e91-51bd-44e2-91c9-f5425a8eae54-ca-trust-extracted\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:56.956633 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956580 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-bound-sa-token\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:56.956633 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956600 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-tls\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:56.956749 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-trusted-ca\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:56.956749 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956680 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/db9e87e7-0480-4019-9416-827502badf63-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-2j44x\" (UID: \"db9e87e7-0480-4019-9416-827502badf63\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" Sep 19 04:16:56.956749 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9248\" (UniqueName: \"kubernetes.io/projected/6b0153cc-54ae-477d-8d6c-cc8677a049e7-kube-api-access-x9248\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:56.956749 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-certificates\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:56.956749 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956740 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkkpj\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-kube-api-access-jkkpj\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:56.956892 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956764 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db9e87e7-0480-4019-9416-827502badf63-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-2j44x\" (UID: \"db9e87e7-0480-4019-9416-827502badf63\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" Sep 19 04:16:56.956925 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956900 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-installation-pull-secrets\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:56.956925 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956918 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-oauth-serving-cert\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:56.956986 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.956937 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-oauth-config\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:56.957055 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.957031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-serving-cert\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:56.957104 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.957079 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-config\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:56.957179 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.957138 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-image-registry-private-configuration\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:56.957344 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.957326 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/db9e87e7-0480-4019-9416-827502badf63-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-2j44x\" (UID: \"db9e87e7-0480-4019-9416-827502badf63\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" Sep 19 04:16:56.959421 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:56.959405 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db9e87e7-0480-4019-9416-827502badf63-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-2j44x\" (UID: \"db9e87e7-0480-4019-9416-827502badf63\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" Sep 19 04:16:57.015506 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.015425 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:57.034273 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.034245 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:16:57.058233 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058205 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-service-ca\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:57.058379 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058243 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28d43e91-51bd-44e2-91c9-f5425a8eae54-ca-trust-extracted\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.058379 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058262 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-bound-sa-token\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.058379 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-tls\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.058379 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058361 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-trusted-ca\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.058577 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058404 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9248\" (UniqueName: \"kubernetes.io/projected/6b0153cc-54ae-477d-8d6c-cc8677a049e7-kube-api-access-x9248\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:57.058577 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-certificates\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.058577 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkkpj\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-kube-api-access-jkkpj\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.058577 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-installation-pull-secrets\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.058577 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058532 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-oauth-serving-cert\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:57.058577 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058559 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-oauth-config\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:57.058857 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058595 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-serving-cert\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:57.058857 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058633 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-config\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:57.058857 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058679 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-image-registry-private-configuration\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.058857 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.058755 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28d43e91-51bd-44e2-91c9-f5425a8eae54-ca-trust-extracted\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.059243 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.059225 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-certificates\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.078372 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.078344 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-bound-sa-token\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.078915 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.078883 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkkpj\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-kube-api-access-jkkpj\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.121610 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.121575 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:57.134399 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.134374 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" Sep 19 04:16:57.239383 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.239345 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:57.288089 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.288063 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xzvch" Sep 19 04:16:57.327546 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.327516 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.330470 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.330443 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn"] Sep 19 04:16:57.334891 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.334870 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.335506 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:57.335483 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ec8341_725a_455e_9039_09136df97735.slice/crio-ed9e44cd300cb1dfcbfdeebaec0dd692e9c5cacce897401d3d66848919a0a198 WatchSource:0}: Error finding container ed9e44cd300cb1dfcbfdeebaec0dd692e9c5cacce897401d3d66848919a0a198: Status 404 returned error can't find the container with id ed9e44cd300cb1dfcbfdeebaec0dd692e9c5cacce897401d3d66848919a0a198 Sep 19 04:16:57.367372 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.367341 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:57.435025 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.434989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-2j44x"] Sep 19 04:16:57.439109 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:57.439088 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9e87e7_0480_4019_9416_827502badf63.slice/crio-f2343fb72ee4e41a728f557343d5d59f94069442537237bf90e3944d6e87c66c WatchSource:0}: Error finding container f2343fb72ee4e41a728f557343d5d59f94069442537237bf90e3944d6e87c66c: Status 404 returned error can't find the container with id f2343fb72ee4e41a728f557343d5d59f94069442537237bf90e3944d6e87c66c Sep 19 04:16:57.461803 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.461774 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkkpj\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-kube-api-access-jkkpj\") pod \"28d43e91-51bd-44e2-91c9-f5425a8eae54\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " Sep 19 04:16:57.461912 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.461818 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28d43e91-51bd-44e2-91c9-f5425a8eae54-ca-trust-extracted\") pod \"28d43e91-51bd-44e2-91c9-f5425a8eae54\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " Sep 19 04:16:57.461912 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.461848 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-certificates\") pod \"28d43e91-51bd-44e2-91c9-f5425a8eae54\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " Sep 19 04:16:57.461912 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.461886 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-bound-sa-token\") pod \"28d43e91-51bd-44e2-91c9-f5425a8eae54\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " Sep 19 04:16:57.462096 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.462065 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d43e91-51bd-44e2-91c9-f5425a8eae54-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "28d43e91-51bd-44e2-91c9-f5425a8eae54" (UID: "28d43e91-51bd-44e2-91c9-f5425a8eae54"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 04:16:57.462217 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.462198 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "28d43e91-51bd-44e2-91c9-f5425a8eae54" (UID: "28d43e91-51bd-44e2-91c9-f5425a8eae54"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:16:57.463997 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.463971 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "28d43e91-51bd-44e2-91c9-f5425a8eae54" (UID: "28d43e91-51bd-44e2-91c9-f5425a8eae54"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:16:57.463997 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.463988 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-kube-api-access-jkkpj" (OuterVolumeSpecName: "kube-api-access-jkkpj") pod "28d43e91-51bd-44e2-91c9-f5425a8eae54" (UID: "28d43e91-51bd-44e2-91c9-f5425a8eae54"). InnerVolumeSpecName "kube-api-access-jkkpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:16:57.563347 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.563273 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkkpj\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-kube-api-access-jkkpj\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:16:57.563347 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.563299 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28d43e91-51bd-44e2-91c9-f5425a8eae54-ca-trust-extracted\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:16:57.563347 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.563309 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-certificates\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:16:57.563347 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.563319 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-bound-sa-token\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:16:57.764711 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.764680 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 19 04:16:57.769727 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.769708 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-service-ca\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:57.807781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.807753 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 19 04:16:57.811045 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.811024 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-oauth-config\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:57.870054 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.869992 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 19 04:16:57.877923 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.877896 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 19 04:16:57.879388 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.879365 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-oauth-serving-cert\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:57.879666 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.879645 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-trusted-ca\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:57.966465 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.966270 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-trusted-ca\") pod \"28d43e91-51bd-44e2-91c9-f5425a8eae54\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " Sep 19 04:16:57.966692 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.966667 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "28d43e91-51bd-44e2-91c9-f5425a8eae54" (UID: "28d43e91-51bd-44e2-91c9-f5425a8eae54"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:16:57.992723 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.992697 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-82l8w\"" Sep 19 04:16:57.997893 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:57.997871 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 19 04:16:58.001355 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.001334 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-tls\") pod \"image-registry-7564fc9dff-c7lgv\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:58.058918 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.058889 2577 configmap.go:193] Couldn't get configMap openshift-console/console-config: failed to sync configmap cache: timed out waiting for the condition Sep 19 04:16:58.058918 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.058902 2577 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: failed to sync secret cache: timed out waiting for the condition Sep 19 04:16:58.058918 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.058917 2577 secret.go:189] Couldn't get secret openshift-image-registry/image-registry-private-configuration: failed to sync secret cache: timed out waiting for the condition Sep 19 04:16:58.059086 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.058969 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-config podName:6b0153cc-54ae-477d-8d6c-cc8677a049e7 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:58.558946416 +0000 UTC m=+84.880044556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-config") pod "console-6b7dfb47d4-246qx" (UID: "6b0153cc-54ae-477d-8d6c-cc8677a049e7") : failed to sync configmap cache: timed out waiting for the condition Sep 19 04:16:58.059086 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.058891 2577 secret.go:189] Couldn't get secret openshift-image-registry/installation-pull-secrets: failed to sync secret cache: timed out waiting for the condition Sep 19 04:16:58.059086 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.058987 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-serving-cert podName:6b0153cc-54ae-477d-8d6c-cc8677a049e7 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:58.558977242 +0000 UTC m=+84.880075368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-serving-cert") pod "console-6b7dfb47d4-246qx" (UID: "6b0153cc-54ae-477d-8d6c-cc8677a049e7") : failed to sync secret cache: timed out waiting for the condition Sep 19 04:16:58.059086 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.059000 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-image-registry-private-configuration podName:28d43e91-51bd-44e2-91c9-f5425a8eae54 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:58.558993569 +0000 UTC m=+84.880091704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "image-registry-private-configuration" (UniqueName: "kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-image-registry-private-configuration") pod "image-registry-7564fc9dff-c7lgv" (UID: "28d43e91-51bd-44e2-91c9-f5425a8eae54") : failed to sync secret cache: timed out waiting for the condition Sep 19 04:16:58.059086 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.059014 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-installation-pull-secrets podName:28d43e91-51bd-44e2-91c9-f5425a8eae54 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:58.559007665 +0000 UTC m=+84.880105799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "installation-pull-secrets" (UniqueName: "kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-installation-pull-secrets") pod "image-registry-7564fc9dff-c7lgv" (UID: "28d43e91-51bd-44e2-91c9-f5425a8eae54") : failed to sync secret cache: timed out waiting for the condition Sep 19 04:16:58.066973 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.066957 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-tls\") pod \"28d43e91-51bd-44e2-91c9-f5425a8eae54\" (UID: \"28d43e91-51bd-44e2-91c9-f5425a8eae54\") " Sep 19 04:16:58.067118 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.067107 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28d43e91-51bd-44e2-91c9-f5425a8eae54-trusted-ca\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:16:58.069061 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.069044 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "28d43e91-51bd-44e2-91c9-f5425a8eae54" (UID: "28d43e91-51bd-44e2-91c9-f5425a8eae54"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:16:58.074960 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.074944 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 19 04:16:58.076869 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.076857 2577 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Sep 19 04:16:58.083070 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.083052 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9h7ml\"" Sep 19 04:16:58.168263 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.168194 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28d43e91-51bd-44e2-91c9-f5425a8eae54-registry-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:16:58.214337 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.214306 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 19 04:16:58.243845 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.243812 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 19 04:16:58.247233 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.247125 2577 projected.go:196] Error preparing data for projected volume kube-api-access-x9248 for pod openshift-console/console-6b7dfb47d4-246qx: failed to sync configmap cache: timed out waiting for the condition Sep 19 04:16:58.247368 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:16:58.247243 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b0153cc-54ae-477d-8d6c-cc8677a049e7-kube-api-access-x9248 podName:6b0153cc-54ae-477d-8d6c-cc8677a049e7 nodeName:}" failed. No retries permitted until 2026-09-19 04:16:58.747210747 +0000 UTC m=+85.068308882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x9248" (UniqueName: "kubernetes.io/projected/6b0153cc-54ae-477d-8d6c-cc8677a049e7-kube-api-access-x9248") pod "console-6b7dfb47d4-246qx" (UID: "6b0153cc-54ae-477d-8d6c-cc8677a049e7") : failed to sync configmap cache: timed out waiting for the condition Sep 19 04:16:58.255273 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.255244 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 19 04:16:58.266994 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.266965 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 19 04:16:58.327754 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.327726 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 19 04:16:58.333241 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.333198 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" event={"ID":"13ec8341-725a-455e-9039-09136df97735","Type":"ContainerStarted","Data":"ed9e44cd300cb1dfcbfdeebaec0dd692e9c5cacce897401d3d66848919a0a198"} Sep 19 04:16:58.335130 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.335104 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7564fc9dff-c7lgv" Sep 19 04:16:58.335278 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.335102 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" event={"ID":"db9e87e7-0480-4019-9416-827502badf63","Type":"ContainerStarted","Data":"f2343fb72ee4e41a728f557343d5d59f94069442537237bf90e3944d6e87c66c"} Sep 19 04:16:58.377277 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.377246 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7564fc9dff-c7lgv"] Sep 19 04:16:58.381917 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.381858 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7564fc9dff-c7lgv"] Sep 19 04:16:58.470395 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.470316 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-image-registry-private-configuration\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:16:58.470395 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.470346 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/28d43e91-51bd-44e2-91c9-f5425a8eae54-installation-pull-secrets\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:16:58.571086 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.571053 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-config\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:58.571232 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.571208 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-serving-cert\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:58.571792 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.571769 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-config\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:58.573754 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.573732 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-serving-cert\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:58.773882 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.773841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9248\" (UniqueName: \"kubernetes.io/projected/6b0153cc-54ae-477d-8d6c-cc8677a049e7-kube-api-access-x9248\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:58.777121 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.777095 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9248\" (UniqueName: \"kubernetes.io/projected/6b0153cc-54ae-477d-8d6c-cc8677a049e7-kube-api-access-x9248\") pod \"console-6b7dfb47d4-246qx\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:58.947637 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:58.947587 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:16:59.341970 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:59.341932 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" event={"ID":"db9e87e7-0480-4019-9416-827502badf63","Type":"ContainerStarted","Data":"05c4ee3fefa1728aafef274d8de7730e376615913cbc68c00eeec9b37aac58d8"} Sep 19 04:16:59.482926 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:59.482883 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:59.585968 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:59.585925 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:59.688106 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:59.687975 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:16:59.894071 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:59.894035 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28d43e91-51bd-44e2-91c9-f5425a8eae54" path="/var/lib/kubelet/pods/28d43e91-51bd-44e2-91c9-f5425a8eae54/volumes" Sep 19 04:16:59.982877 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:59.982764 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-2j44x" podStartSLOduration=2.858167263 podStartE2EDuration="3.982744025s" totalImagesPullingTime="1.124576762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:16:59.360762225 +0000 UTC m=+85.681860371" watchObservedRunningTime="2026-09-19 04:16:59.982744025 +0000 UTC m=+86.303842161" Sep 19 04:16:59.984672 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:16:59.983834 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7dfb47d4-246qx"] Sep 19 04:16:59.988329 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:16:59.988299 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b0153cc_54ae_477d_8d6c_cc8677a049e7.slice/crio-84b7af8d8f2832a34d188cd91c1d992a9decfaa92f0eb2391ad9f0d8812acfc6 WatchSource:0}: Error finding container 84b7af8d8f2832a34d188cd91c1d992a9decfaa92f0eb2391ad9f0d8812acfc6: Status 404 returned error can't find the container with id 84b7af8d8f2832a34d188cd91c1d992a9decfaa92f0eb2391ad9f0d8812acfc6 Sep 19 04:17:00.350751 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:00.350711 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7dfb47d4-246qx" event={"ID":"6b0153cc-54ae-477d-8d6c-cc8677a049e7","Type":"ContainerStarted","Data":"84b7af8d8f2832a34d188cd91c1d992a9decfaa92f0eb2391ad9f0d8812acfc6"} Sep 19 04:17:01.249097 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:01.249061 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:01.328101 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:01.328059 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:02.361221 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:02.361167 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" event={"ID":"13ec8341-725a-455e-9039-09136df97735","Type":"ContainerStarted","Data":"6b80a0d1139a9ce0136b123a8e0080a4a47fdf853cb09992ae367b5b72bbd306"} Sep 19 04:17:02.361664 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:02.361360 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:17:02.363085 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:02.363060 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" Sep 19 04:17:02.385322 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:02.383873 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c6d9d8c86-dm5pn" podStartSLOduration=2.3982439380000002 podStartE2EDuration="6.383854103s" totalImagesPullingTime="3.985610165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:02.382664664 +0000 UTC m=+88.703762811" watchObservedRunningTime="2026-09-19 04:17:02.383854103 +0000 UTC m=+88.704952251" Sep 19 04:17:03.383426 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.383388 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:03.490972 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.490927 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:03.614419 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.614381 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x"] Sep 19 04:17:03.628793 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.628756 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.635735 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.635657 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 19 04:17:03.635892 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.635812 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 19 04:17:03.636910 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.636888 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 19 04:17:03.637098 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.637082 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-n25bj\"" Sep 19 04:17:03.637323 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.637296 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 19 04:17:03.644202 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.644183 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 19 04:17:03.644683 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.644661 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x"] Sep 19 04:17:03.662489 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.662456 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:03.715308 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.715268 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/610ed2d1-470b-4c8d-973f-688c6de8053e-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.715452 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.715322 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/610ed2d1-470b-4c8d-973f-688c6de8053e-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.715452 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.715389 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/610ed2d1-470b-4c8d-973f-688c6de8053e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.715452 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.715448 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9s9q\" (UniqueName: \"kubernetes.io/projected/610ed2d1-470b-4c8d-973f-688c6de8053e-kube-api-access-j9s9q\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.816633 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.816589 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/610ed2d1-470b-4c8d-973f-688c6de8053e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.816821 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.816652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9s9q\" (UniqueName: \"kubernetes.io/projected/610ed2d1-470b-4c8d-973f-688c6de8053e-kube-api-access-j9s9q\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.816821 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.816696 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/610ed2d1-470b-4c8d-973f-688c6de8053e-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.816821 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.816729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/610ed2d1-470b-4c8d-973f-688c6de8053e-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.817561 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.817527 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/610ed2d1-470b-4c8d-973f-688c6de8053e-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.819577 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.819557 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/610ed2d1-470b-4c8d-973f-688c6de8053e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.819697 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.819648 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/610ed2d1-470b-4c8d-973f-688c6de8053e-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.827470 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.827444 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9s9q\" (UniqueName: \"kubernetes.io/projected/610ed2d1-470b-4c8d-973f-688c6de8053e-kube-api-access-j9s9q\") pod \"prometheus-operator-7bd5d56d6-knh8x\" (UID: \"610ed2d1-470b-4c8d-973f-688c6de8053e\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:03.938911 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:03.938824 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" Sep 19 04:17:05.061960 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:05.061870 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x"] Sep 19 04:17:05.065043 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:17:05.065014 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610ed2d1_470b_4c8d_973f_688c6de8053e.slice/crio-c0dec314f9ecdda2638bb8637eff0bf845e997925d73aaf8cbf888604a9a8b29 WatchSource:0}: Error finding container c0dec314f9ecdda2638bb8637eff0bf845e997925d73aaf8cbf888604a9a8b29: Status 404 returned error can't find the container with id c0dec314f9ecdda2638bb8637eff0bf845e997925d73aaf8cbf888604a9a8b29 Sep 19 04:17:05.374496 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:05.374389 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7dfb47d4-246qx" event={"ID":"6b0153cc-54ae-477d-8d6c-cc8677a049e7","Type":"ContainerStarted","Data":"a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4"} Sep 19 04:17:05.375518 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:05.375490 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" event={"ID":"610ed2d1-470b-4c8d-973f-688c6de8053e","Type":"ContainerStarted","Data":"c0dec314f9ecdda2638bb8637eff0bf845e997925d73aaf8cbf888604a9a8b29"} Sep 19 04:17:05.408281 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:05.408243 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:05.507593 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:05.507553 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:06.668736 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:06.668706 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:06.825267 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:06.825137 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:07.003462 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:07.003415 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:07.384761 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:07.384721 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" event={"ID":"610ed2d1-470b-4c8d-973f-688c6de8053e","Type":"ContainerStarted","Data":"aeaf9b24cf410d1515cc95a2f3f9f335f6d93b7e2bc90843a83290a12eb61182"} Sep 19 04:17:07.384953 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:07.384788 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" event={"ID":"610ed2d1-470b-4c8d-973f-688c6de8053e","Type":"ContainerStarted","Data":"7cbd0fde0b8c970ffea90d191442c702d30e2f06aa2425029879f11895867ce5"} Sep 19 04:17:07.436078 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:07.436026 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b7dfb47d4-246qx" podStartSLOduration=6.583434939 podStartE2EDuration="11.436009076s" totalImagesPullingTime="4.852574137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:05.397439673 +0000 UTC m=+91.718537819" watchObservedRunningTime="2026-09-19 04:17:07.436009076 +0000 UTC m=+93.757107240" Sep 19 04:17:07.436248 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:07.436128 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-knh8x" podStartSLOduration=2.966931744 podStartE2EDuration="4.436120786s" totalImagesPullingTime="1.469189042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:07.435000962 +0000 UTC m=+93.756099100" watchObservedRunningTime="2026-09-19 04:17:07.436120786 +0000 UTC m=+93.757218932" Sep 19 04:17:08.778437 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:08.778400 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:08.948506 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:08.948474 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:17:08.948679 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:08.948531 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:17:08.956782 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:08.956752 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:17:09.161478 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.161395 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv"] Sep 19 04:17:09.165040 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.165012 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.165354 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.165327 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9cxmb"] Sep 19 04:17:09.168495 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.168478 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.169076 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.169057 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 19 04:17:09.169146 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.169115 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 19 04:17:09.169244 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.169214 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f6jmd\"" Sep 19 04:17:09.172840 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.172822 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 19 04:17:09.173025 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.173006 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 19 04:17:09.173115 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.173043 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vjdsw\"" Sep 19 04:17:09.173330 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.173311 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 19 04:17:09.179006 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.178983 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:09.189649 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.189628 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv"] Sep 19 04:17:09.222009 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.221971 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-k89dx"] Sep 19 04:17:09.225504 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.225485 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.228607 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.228584 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 19 04:17:09.229133 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.229113 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5vq6l\"" Sep 19 04:17:09.229276 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.229259 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 19 04:17:09.229418 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.229402 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 19 04:17:09.249417 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.249396 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-k89dx"] Sep 19 04:17:09.262350 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262327 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/805b7bf9-b662-400b-a35a-d52ff284ca23-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.262447 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-wtmp\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.262447 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/249ddd11-73ec-42f8-8212-c7d7110de056-sys\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.262447 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262400 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/249ddd11-73ec-42f8-8212-c7d7110de056-metrics-client-ca\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.262447 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262428 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/805b7bf9-b662-400b-a35a-d52ff284ca23-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.262447 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262446 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhfhm\" (UniqueName: \"kubernetes.io/projected/805b7bf9-b662-400b-a35a-d52ff284ca23-kube-api-access-mhfhm\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.262645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262465 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/249ddd11-73ec-42f8-8212-c7d7110de056-root\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.262645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262482 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-textfile\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.262645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-tls\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.262645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-accelerators-collector-config\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.262781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262642 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.262781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/805b7bf9-b662-400b-a35a-d52ff284ca23-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.262781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.262757 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49bc2\" (UniqueName: \"kubernetes.io/projected/249ddd11-73ec-42f8-8212-c7d7110de056-kube-api-access-49bc2\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.363865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.363830 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc4411b4-df39-4284-bba8-a14812163819-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.364063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.363885 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/805b7bf9-b662-400b-a35a-d52ff284ca23-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.364063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.363920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49bc2\" (UniqueName: \"kubernetes.io/projected/249ddd11-73ec-42f8-8212-c7d7110de056-kube-api-access-49bc2\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.363943 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bljq8\" (UniqueName: \"kubernetes.io/projected/cc4411b4-df39-4284-bba8-a14812163819-kube-api-access-bljq8\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.364063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.363967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cc4411b4-df39-4284-bba8-a14812163819-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.364287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364097 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/805b7bf9-b662-400b-a35a-d52ff284ca23-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.364287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364143 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-wtmp\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364208 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/249ddd11-73ec-42f8-8212-c7d7110de056-sys\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/249ddd11-73ec-42f8-8212-c7d7110de056-metrics-client-ca\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364287 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/805b7bf9-b662-400b-a35a-d52ff284ca23-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.364512 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhfhm\" (UniqueName: \"kubernetes.io/projected/805b7bf9-b662-400b-a35a-d52ff284ca23-kube-api-access-mhfhm\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.364512 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364329 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/249ddd11-73ec-42f8-8212-c7d7110de056-root\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364512 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364333 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/249ddd11-73ec-42f8-8212-c7d7110de056-sys\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364512 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364381 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-wtmp\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364512 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:17:09.364399 2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 19 04:17:09.364512 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-textfile\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364512 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/249ddd11-73ec-42f8-8212-c7d7110de056-root\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364512 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364442 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cc4411b4-df39-4284-bba8-a14812163819-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.364512 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:17:09.364465 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/805b7bf9-b662-400b-a35a-d52ff284ca23-openshift-state-metrics-tls podName:805b7bf9-b662-400b-a35a-d52ff284ca23 nodeName:}" failed. No retries permitted until 2026-09-19 04:17:09.864445997 +0000 UTC m=+96.185544146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/805b7bf9-b662-400b-a35a-d52ff284ca23-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-z8jkv" (UID: "805b7bf9-b662-400b-a35a-d52ff284ca23") : secret "openshift-state-metrics-tls" not found Sep 19 04:17:09.364898 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc4411b4-df39-4284-bba8-a14812163819-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.364898 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc4411b4-df39-4284-bba8-a14812163819-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.364898 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364624 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-tls\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364898 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-accelerators-collector-config\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364898 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364898 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:17:09.364707 2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 19 04:17:09.364898 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-textfile\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.364898 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.364706 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/805b7bf9-b662-400b-a35a-d52ff284ca23-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.364898 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:17:09.364753 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-tls podName:249ddd11-73ec-42f8-8212-c7d7110de056 nodeName:}" failed. No retries permitted until 2026-09-19 04:17:09.864736299 +0000 UTC m=+96.185834436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-tls") pod "node-exporter-9cxmb" (UID: "249ddd11-73ec-42f8-8212-c7d7110de056") : secret "node-exporter-tls" not found Sep 19 04:17:09.365244 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.365216 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-accelerators-collector-config\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.365244 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.365238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/249ddd11-73ec-42f8-8212-c7d7110de056-metrics-client-ca\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.366747 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.366723 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/805b7bf9-b662-400b-a35a-d52ff284ca23-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.367010 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.366988 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.375953 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.375919 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:09.394837 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.394812 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:17:09.397389 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.397361 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49bc2\" (UniqueName: \"kubernetes.io/projected/249ddd11-73ec-42f8-8212-c7d7110de056-kube-api-access-49bc2\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.398516 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.398499 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhfhm\" (UniqueName: \"kubernetes.io/projected/805b7bf9-b662-400b-a35a-d52ff284ca23-kube-api-access-mhfhm\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.465179 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.465074 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bljq8\" (UniqueName: \"kubernetes.io/projected/cc4411b4-df39-4284-bba8-a14812163819-kube-api-access-bljq8\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.465179 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.465110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cc4411b4-df39-4284-bba8-a14812163819-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.465416 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.465354 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cc4411b4-df39-4284-bba8-a14812163819-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.465416 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.465394 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc4411b4-df39-4284-bba8-a14812163819-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.465519 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.465436 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc4411b4-df39-4284-bba8-a14812163819-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.465570 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.465529 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc4411b4-df39-4284-bba8-a14812163819-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.465763 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.465733 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cc4411b4-df39-4284-bba8-a14812163819-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.465877 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.465861 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cc4411b4-df39-4284-bba8-a14812163819-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.465960 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.465943 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc4411b4-df39-4284-bba8-a14812163819-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.467908 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.467888 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc4411b4-df39-4284-bba8-a14812163819-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.468069 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.468049 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc4411b4-df39-4284-bba8-a14812163819-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.513807 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.513775 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bljq8\" (UniqueName: \"kubernetes.io/projected/cc4411b4-df39-4284-bba8-a14812163819-kube-api-access-bljq8\") pod \"kube-state-metrics-84c885658c-k89dx\" (UID: \"cc4411b4-df39-4284-bba8-a14812163819\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.534667 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.534635 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" Sep 19 04:17:09.567527 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.567496 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:09.691869 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.691830 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:09.805515 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.805459 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:09.823432 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.823403 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-k89dx"] Sep 19 04:17:09.826002 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:17:09.825966 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc4411b4_df39_4284_bba8_a14812163819.slice/crio-4b390decd3292d53847708ad40d73ed5788d342b4edb7c7312c6f1d6c47aed17 WatchSource:0}: Error finding container 4b390decd3292d53847708ad40d73ed5788d342b4edb7c7312c6f1d6c47aed17: Status 404 returned error can't find the container with id 4b390decd3292d53847708ad40d73ed5788d342b4edb7c7312c6f1d6c47aed17 Sep 19 04:17:09.869744 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.869708 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/805b7bf9-b662-400b-a35a-d52ff284ca23-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.869889 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.869775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-tls\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.872328 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.872295 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/249ddd11-73ec-42f8-8212-c7d7110de056-node-exporter-tls\") pod \"node-exporter-9cxmb\" (UID: \"249ddd11-73ec-42f8-8212-c7d7110de056\") " pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:09.872413 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.872384 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/805b7bf9-b662-400b-a35a-d52ff284ca23-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-z8jkv\" (UID: \"805b7bf9-b662-400b-a35a-d52ff284ca23\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:09.923940 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:09.923892 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:10.075346 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.075258 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" Sep 19 04:17:10.080769 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.080744 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9cxmb" Sep 19 04:17:10.092404 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:17:10.092373 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod249ddd11_73ec_42f8_8212_c7d7110de056.slice/crio-5155c03ebe4a69b0cb522de9405b3a748230dbc592d7f3549529bddd6c3b1f39 WatchSource:0}: Error finding container 5155c03ebe4a69b0cb522de9405b3a748230dbc592d7f3549529bddd6c3b1f39: Status 404 returned error can't find the container with id 5155c03ebe4a69b0cb522de9405b3a748230dbc592d7f3549529bddd6c3b1f39 Sep 19 04:17:10.152616 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.152582 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:10.254621 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.254567 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:10.362922 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.362836 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:10.387185 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.387138 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv"] Sep 19 04:17:10.391052 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:17:10.391022 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805b7bf9_b662_400b_a35a_d52ff284ca23.slice/crio-7ed637539b2cd6c539d7c6503f918659af243ce8aae09d46bc53173d63d0081f WatchSource:0}: Error finding container 7ed637539b2cd6c539d7c6503f918659af243ce8aae09d46bc53173d63d0081f: Status 404 returned error can't find the container with id 7ed637539b2cd6c539d7c6503f918659af243ce8aae09d46bc53173d63d0081f Sep 19 04:17:10.399485 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.399401 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" event={"ID":"cc4411b4-df39-4284-bba8-a14812163819","Type":"ContainerStarted","Data":"4b390decd3292d53847708ad40d73ed5788d342b4edb7c7312c6f1d6c47aed17"} Sep 19 04:17:10.402831 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.402801 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9cxmb" event={"ID":"249ddd11-73ec-42f8-8212-c7d7110de056","Type":"ContainerStarted","Data":"5155c03ebe4a69b0cb522de9405b3a748230dbc592d7f3549529bddd6c3b1f39"} Sep 19 04:17:10.404657 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.404587 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" event={"ID":"805b7bf9-b662-400b-a35a-d52ff284ca23","Type":"ContainerStarted","Data":"7ed637539b2cd6c539d7c6503f918659af243ce8aae09d46bc53173d63d0081f"} Sep 19 04:17:10.450924 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.450885 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 04:17:10.456381 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.456353 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.465662 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.465639 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 19 04:17:10.465818 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.465795 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 19 04:17:10.465913 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.465891 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 19 04:17:10.466420 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.466104 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 19 04:17:10.466420 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.466172 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zh2j7\"" Sep 19 04:17:10.466420 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.466283 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 19 04:17:10.466420 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.466306 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 19 04:17:10.466420 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.466387 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 19 04:17:10.466420 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.466421 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 19 04:17:10.472004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.471984 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 19 04:17:10.484032 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.483898 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 04:17:10.499691 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.499612 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:10.578076 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578044 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578264 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578083 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578264 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578114 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-config-out\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578264 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578136 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578264 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578251 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578493 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578302 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578493 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578324 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578493 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578379 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578493 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578493 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578453 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-web-config\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578524 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578560 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-config-volume\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.578645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.578580 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5btw\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-kube-api-access-c5btw\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.679856 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.679762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.679856 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.679815 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.679856 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.679847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-config-out\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.679872 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.679909 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.679940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.679970 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.680005 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.680033 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.680104 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-web-config\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.680148 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.680188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-config-volume\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.680206 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5btw\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-kube-api-access-c5btw\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.680942 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.680909 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.681770 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.681493 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.681891 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.681816 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.683944 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.683881 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.687592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.687563 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-config-volume\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.688822 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.688774 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.689665 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.689638 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.690737 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.690291 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.690737 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.690485 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-config-out\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.690737 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.690696 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.692470 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.692432 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-web-config\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.692554 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.692472 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.697698 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.697679 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5btw\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-kube-api-access-c5btw\") pod \"alertmanager-main-0\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.768020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.767987 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:17:10.798688 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:10.798452 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:11.096192 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:11.096143 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 04:17:11.136409 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:17:11.136367 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952c8259_3d05_45a7_aa71_3278a80f92f8.slice/crio-24d35c2f67b4274929f65962c8b3c65250f47ab60a3fbafc668ed39e2ed99ed6 WatchSource:0}: Error finding container 24d35c2f67b4274929f65962c8b3c65250f47ab60a3fbafc668ed39e2ed99ed6: Status 404 returned error can't find the container with id 24d35c2f67b4274929f65962c8b3c65250f47ab60a3fbafc668ed39e2ed99ed6 Sep 19 04:17:11.409740 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:11.409708 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerStarted","Data":"24d35c2f67b4274929f65962c8b3c65250f47ab60a3fbafc668ed39e2ed99ed6"} Sep 19 04:17:11.411972 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:11.411945 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" event={"ID":"805b7bf9-b662-400b-a35a-d52ff284ca23","Type":"ContainerStarted","Data":"5b717e200389888c2347ffc28de6fa884f0c0300debc44b6ec0f0f5eebc755f8"} Sep 19 04:17:11.412091 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:11.412007 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" event={"ID":"805b7bf9-b662-400b-a35a-d52ff284ca23","Type":"ContainerStarted","Data":"143d3653b722947a3b30a62851dc28e7dca86596d7456a48c70f7bd7e98fbd2f"} Sep 19 04:17:12.395539 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.395508 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:12.421743 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.421708 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" event={"ID":"cc4411b4-df39-4284-bba8-a14812163819","Type":"ContainerStarted","Data":"b27b1dca0f3a5797bcef487ca42e21c60a539110446282f8b5156eca480ef6c0"} Sep 19 04:17:12.421873 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.421769 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" event={"ID":"cc4411b4-df39-4284-bba8-a14812163819","Type":"ContainerStarted","Data":"c6bbfedad152f251dcaa69ad266ae857d4d6369fc6ad97fb4ff29b53b0601050"} Sep 19 04:17:12.421873 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.421800 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" event={"ID":"cc4411b4-df39-4284-bba8-a14812163819","Type":"ContainerStarted","Data":"a89bc4eb457786c6313f7486cedb4fde05a6733d99cded1089c4b1068a1c1f39"} Sep 19 04:17:12.423367 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.423341 2577 generic.go:356] "Generic (PLEG): container finished" podID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerID="87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d" exitCode=0 Sep 19 04:17:12.423476 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.423425 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerDied","Data":"87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d"} Sep 19 04:17:12.425219 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.425191 2577 generic.go:356] "Generic (PLEG): container finished" podID="249ddd11-73ec-42f8-8212-c7d7110de056" containerID="efbfc7a105e174af909f12c4ca398dc6b4a5063fe85bca84ebd483725c9c7861" exitCode=0 Sep 19 04:17:12.425360 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.425269 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9cxmb" event={"ID":"249ddd11-73ec-42f8-8212-c7d7110de056","Type":"ContainerDied","Data":"efbfc7a105e174af909f12c4ca398dc6b4a5063fe85bca84ebd483725c9c7861"} Sep 19 04:17:12.427674 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.427650 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" event={"ID":"805b7bf9-b662-400b-a35a-d52ff284ca23","Type":"ContainerStarted","Data":"84687a3e17ace1d74f48f58708c749a58c544d7adfab7ca0b9415a1195b72c35"} Sep 19 04:17:12.487993 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.487948 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-k89dx" podStartSLOduration=1.911877324 podStartE2EDuration="3.487932003s" totalImagesPullingTime="1.576054679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:12.483425249 +0000 UTC m=+98.804523406" watchObservedRunningTime="2026-09-19 04:17:12.487932003 +0000 UTC m=+98.809030149" Sep 19 04:17:12.569805 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.569772 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:12.589821 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.589771 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-z8jkv" podStartSLOduration=2.2410018640000002 podStartE2EDuration="3.589754668s" totalImagesPullingTime="1.348752804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:12.532857407 +0000 UTC m=+98.853955553" watchObservedRunningTime="2026-09-19 04:17:12.589754668 +0000 UTC m=+98.910852811" Sep 19 04:17:12.696034 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:12.695995 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:13.043564 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:13.043524 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:13.295050 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:13.294942 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:13.413138 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:13.413106 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:13.433647 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:13.433611 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9cxmb" event={"ID":"249ddd11-73ec-42f8-8212-c7d7110de056","Type":"ContainerStarted","Data":"ac1b0931043172d534112fa761098ae8a557bf265ca3defc8c5844869e9fae39"} Sep 19 04:17:13.433790 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:13.433709 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9cxmb" event={"ID":"249ddd11-73ec-42f8-8212-c7d7110de056","Type":"ContainerStarted","Data":"28e6a73ed1e98a775a04a91667802b41dc0b8c4bada6ffc6b4f349155ad83be6"} Sep 19 04:17:13.554385 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:13.554296 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:13.911063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:13.911028 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:14.026945 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.026901 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9cxmb" podStartSLOduration=3.62877317 podStartE2EDuration="5.0268841s" totalImagesPullingTime="1.312153987s" totalInitContainerRuntime="85.956943ms" isStatefulPod=true podCreationTimestamp="2026-09-19 04:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:13.517474298 +0000 UTC m=+99.838572444" watchObservedRunningTime="2026-09-19 04:17:14.0268841 +0000 UTC m=+100.347982245" Sep 19 04:17:14.027820 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.027799 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5"] Sep 19 04:17:14.031301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.031281 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.036163 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.036124 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 19 04:17:14.036271 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.036124 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 19 04:17:14.036271 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.036183 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 19 04:17:14.036271 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.036194 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-s99h5\"" Sep 19 04:17:14.036271 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.036249 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dq4bi2kio3koo\"" Sep 19 04:17:14.036677 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.036662 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 19 04:17:14.056118 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.056071 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5"] Sep 19 04:17:14.072652 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.072622 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:14.111788 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.111757 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-client-ca-bundle\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.111920 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.111799 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.111920 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.111872 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcwdr\" (UniqueName: \"kubernetes.io/projected/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-kube-api-access-gcwdr\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.111997 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.111919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-secret-metrics-server-tls\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.112030 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.112009 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-secret-metrics-server-client-certs\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.112062 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.112049 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-audit-log\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.112100 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.112076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-metrics-server-audit-profiles\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.212514 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.212474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-secret-metrics-server-client-certs\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.212514 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.212513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-audit-log\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.212761 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.212625 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-metrics-server-audit-profiles\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.212761 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.212700 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-client-ca-bundle\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.212761 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.212731 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.212916 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.212778 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcwdr\" (UniqueName: \"kubernetes.io/projected/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-kube-api-access-gcwdr\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.212916 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.212840 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-audit-log\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.212916 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.212860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-secret-metrics-server-tls\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.213497 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.213474 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.213885 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.213861 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-metrics-server-audit-profiles\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.215142 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.215120 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-client-ca-bundle\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.215247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.215186 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-secret-metrics-server-client-certs\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.215247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.215218 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-secret-metrics-server-tls\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.255638 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.255613 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcwdr\" (UniqueName: \"kubernetes.io/projected/c61fb3ac-0fc4-487a-9fe8-d3f74a660f81-kube-api-access-gcwdr\") pod \"metrics-server-6dfd8fc58b-tq5h5\" (UID: \"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81\") " pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.294255 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.294223 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:14.345900 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.345826 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:14.440801 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.440764 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerStarted","Data":"8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d"} Sep 19 04:17:14.441140 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.440884 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerStarted","Data":"874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98"} Sep 19 04:17:14.441140 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.440930 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerStarted","Data":"739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441"} Sep 19 04:17:14.441140 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.440972 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerStarted","Data":"c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3"} Sep 19 04:17:14.441140 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.441001 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerStarted","Data":"9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b"} Sep 19 04:17:14.660241 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:14.654926 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5"] Sep 19 04:17:14.661277 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:17:14.661213 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61fb3ac_0fc4_487a_9fe8_d3f74a660f81.slice/crio-378cbf68ac46e36997670dcef6ec70e32398d0d5ce5279b1fda818e5e1e5e853 WatchSource:0}: Error finding container 378cbf68ac46e36997670dcef6ec70e32398d0d5ce5279b1fda818e5e1e5e853: Status 404 returned error can't find the container with id 378cbf68ac46e36997670dcef6ec70e32398d0d5ce5279b1fda818e5e1e5e853 Sep 19 04:17:15.118893 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.118854 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:15.264812 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.264722 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:15.315875 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.315840 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-699778cc4b-9cmcd"] Sep 19 04:17:15.319302 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.319271 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.331848 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.331823 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 19 04:17:15.351929 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.351898 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-699778cc4b-9cmcd"] Sep 19 04:17:15.424098 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.424057 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-serving-cert\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.424316 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.424123 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-oauth-serving-cert\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.424316 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.424172 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-oauth-config\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.424316 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.424201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-service-ca\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.424316 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.424287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tsft\" (UniqueName: \"kubernetes.io/projected/fd09ca20-dece-4514-8a03-1b4f13f22789-kube-api-access-7tsft\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.424513 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.424321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-trusted-ca-bundle\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.424513 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.424360 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-console-config\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.450456 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.450355 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerStarted","Data":"b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec"} Sep 19 04:17:15.452015 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.451989 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" event={"ID":"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81","Type":"ContainerStarted","Data":"378cbf68ac46e36997670dcef6ec70e32398d0d5ce5279b1fda818e5e1e5e853"} Sep 19 04:17:15.522681 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.522627 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.228713165 podStartE2EDuration="5.522602574s" totalImagesPullingTime="3.277558427s" totalInitContainerRuntime="16.330982ms" isStatefulPod=false podCreationTimestamp="2026-09-19 04:17:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:15.51701216 +0000 UTC m=+101.838110318" watchObservedRunningTime="2026-09-19 04:17:15.522602574 +0000 UTC m=+101.843700733" Sep 19 04:17:15.525266 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.525239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tsft\" (UniqueName: \"kubernetes.io/projected/fd09ca20-dece-4514-8a03-1b4f13f22789-kube-api-access-7tsft\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.525492 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.525471 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-trusted-ca-bundle\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.525583 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.525559 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-console-config\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.525821 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.525799 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-serving-cert\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.525905 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.525878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-oauth-serving-cert\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.525972 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.525923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-oauth-config\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.525972 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.525960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-service-ca\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.526538 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.526493 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-trusted-ca-bundle\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.526629 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.526555 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-console-config\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.526629 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.526604 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-service-ca\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.526830 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.526808 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-oauth-serving-cert\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.528787 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.528764 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-oauth-config\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.529201 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.529178 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-serving-cert\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.558424 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.558384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:15.568546 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.568483 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tsft\" (UniqueName: \"kubernetes.io/projected/fd09ca20-dece-4514-8a03-1b4f13f22789-kube-api-access-7tsft\") pod \"console-699778cc4b-9cmcd\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.630300 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.630268 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:15.746399 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:15.746356 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:16.105476 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.105430 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:16.129941 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.129904 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 04:17:16.136597 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.136572 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.157622 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.157567 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 19 04:17:16.165500 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.165430 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 19 04:17:16.165645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.165432 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 19 04:17:16.165645 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.165595 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 19 04:17:16.165748 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.165667 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 19 04:17:16.165853 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.165837 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e6dr04e4804kb\"" Sep 19 04:17:16.165929 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.165909 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 19 04:17:16.165980 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.165919 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-x2fn9\"" Sep 19 04:17:16.166063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.166049 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 19 04:17:16.166251 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.166223 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 19 04:17:16.166385 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.166369 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 19 04:17:16.166455 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.166436 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 19 04:17:16.166717 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.166703 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 19 04:17:16.167175 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.167131 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 19 04:17:16.190552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.190514 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 19 04:17:16.193818 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.193792 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 19 04:17:16.221723 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.221654 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-699778cc4b-9cmcd"] Sep 19 04:17:16.223998 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:17:16.223968 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd09ca20_dece_4514_8a03_1b4f13f22789.slice/crio-6ad222bd4113a000baca8822e581dc3b004097da9f2aa539c5f36bf7953771f1 WatchSource:0}: Error finding container 6ad222bd4113a000baca8822e581dc3b004097da9f2aa539c5f36bf7953771f1: Status 404 returned error can't find the container with id 6ad222bd4113a000baca8822e581dc3b004097da9f2aa539c5f36bf7953771f1 Sep 19 04:17:16.224088 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.223998 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 04:17:16.232685 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.232663 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.232762 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.232716 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.232818 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.232774 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.232866 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.232818 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.232906 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.232874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.232943 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.232905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.232987 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.232939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233027 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.232999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233071 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233071 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233060 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233144 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233084 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tngc5\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-kube-api-access-tngc5\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233144 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233109 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233144 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233137 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233231 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-config\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233289 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233257 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233338 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233501 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233401 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.233501 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.233442 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.334599 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.334552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-config\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.334599 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.334587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.334800 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.334766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.334866 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.334821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.334866 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.334852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.334972 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.334888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.334972 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.334928 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.334972 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.334959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335111 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335018 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335111 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335111 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335114 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335225 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335264 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335293 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335576 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335316 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335576 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335343 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tngc5\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-kube-api-access-tngc5\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335576 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335371 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.335576 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.335402 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.336135 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.336109 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.336294 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.336269 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.336477 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.336456 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.338181 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.338130 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.339370 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.338460 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.339370 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.338881 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.339370 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.339043 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.339370 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.339238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.339982 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.339954 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.340470 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.340425 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.340919 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.340892 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.341862 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.341817 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.342449 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.342428 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.343207 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.342848 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.343207 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.342982 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.343207 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.343012 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.343207 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.343090 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.347398 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.347359 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.347506 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.343180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-config\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.354758 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.354734 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tngc5\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-kube-api-access-tngc5\") pod \"prometheus-k8s-0\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.446957 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.446869 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:16.459063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.459018 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-699778cc4b-9cmcd" event={"ID":"fd09ca20-dece-4514-8a03-1b4f13f22789","Type":"ContainerStarted","Data":"06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53"} Sep 19 04:17:16.459546 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.459089 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-699778cc4b-9cmcd" event={"ID":"fd09ca20-dece-4514-8a03-1b4f13f22789","Type":"ContainerStarted","Data":"6ad222bd4113a000baca8822e581dc3b004097da9f2aa539c5f36bf7953771f1"} Sep 19 04:17:16.461564 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.461527 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" event={"ID":"c61fb3ac-0fc4-487a-9fe8-d3f74a660f81","Type":"ContainerStarted","Data":"2198bbb6ec03a03ab823e9352121d4a224d161e6fb845b39616fde5769a04594"} Sep 19 04:17:16.495282 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.495219 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-699778cc4b-9cmcd" podStartSLOduration=1.495196658 podStartE2EDuration="1.495196658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:17:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:16.493809342 +0000 UTC m=+102.814907488" watchObservedRunningTime="2026-09-19 04:17:16.495196658 +0000 UTC m=+102.816294806" Sep 19 04:17:16.804795 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.804730 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" podStartSLOduration=2.510324014 podStartE2EDuration="3.804709001s" totalImagesPullingTime="1.294384987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:16.567579312 +0000 UTC m=+102.888677469" watchObservedRunningTime="2026-09-19 04:17:16.804709001 +0000 UTC m=+103.125807146" Sep 19 04:17:16.805597 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:16.805576 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 04:17:16.808790 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:17:16.808764 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796ea408_c763_48f1_80ca_8e21977b17b7.slice/crio-766cf9fb5630b781da7c6ca4c6bdd2b695bc048c133a13108698caca4f23f16c WatchSource:0}: Error finding container 766cf9fb5630b781da7c6ca4c6bdd2b695bc048c133a13108698caca4f23f16c: Status 404 returned error can't find the container with id 766cf9fb5630b781da7c6ca4c6bdd2b695bc048c133a13108698caca4f23f16c Sep 19 04:17:17.467282 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:17.467250 2577 generic.go:356] "Generic (PLEG): container finished" podID="796ea408-c763-48f1-80ca-8e21977b17b7" containerID="6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d" exitCode=0 Sep 19 04:17:17.467823 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:17.467338 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerDied","Data":"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d"} Sep 19 04:17:17.467823 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:17.467425 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerStarted","Data":"766cf9fb5630b781da7c6ca4c6bdd2b695bc048c133a13108698caca4f23f16c"} Sep 19 04:17:17.592380 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:17.592349 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:17.769984 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:17.769947 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:17.966468 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:17.966426 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:18.099199 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:18.099091 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:18.288917 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:18.288884 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nqvp6" Sep 19 04:17:19.135137 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:19.135101 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:19.250936 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:19.250897 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:19.882912 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:19.882872 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:20.042000 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:20.041959 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:20.282429 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:20.282391 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:20.398900 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:20.398828 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:21.490639 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:21.490606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerStarted","Data":"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c"} Sep 19 04:17:21.491000 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:21.490669 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerStarted","Data":"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e"} Sep 19 04:17:23.506883 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:23.506846 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerStarted","Data":"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5"} Sep 19 04:17:23.507346 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:23.506959 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerStarted","Data":"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec"} Sep 19 04:17:23.507346 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:23.507006 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerStarted","Data":"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3"} Sep 19 04:17:23.507346 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:23.507050 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerStarted","Data":"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269"} Sep 19 04:17:23.583845 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:23.583791 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.6535591480000003 podStartE2EDuration="7.583773801s" totalImagesPullingTime="4.909164905s" totalInitContainerRuntime="21.049748ms" isStatefulPod=false podCreationTimestamp="2026-09-19 04:17:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:17:23.582003887 +0000 UTC m=+109.903102070" watchObservedRunningTime="2026-09-19 04:17:23.583773801 +0000 UTC m=+109.904871948" Sep 19 04:17:25.516604 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:25.516569 2577 generic.go:356] "Generic (PLEG): container finished" podID="0f534b12-a447-453d-aa10-9118ac1b6d53" containerID="4ae7bcde029003d978721036fade5e4649002324c5938d0728f59e20f4dce769" exitCode=0 Sep 19 04:17:25.516965 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:25.516642 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" event={"ID":"0f534b12-a447-453d-aa10-9118ac1b6d53","Type":"ContainerDied","Data":"4ae7bcde029003d978721036fade5e4649002324c5938d0728f59e20f4dce769"} Sep 19 04:17:25.517011 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:25.517003 2577 scope.go:117] "RemoveContainer" containerID="4ae7bcde029003d978721036fade5e4649002324c5938d0728f59e20f4dce769" Sep 19 04:17:25.630366 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:25.630336 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:25.630500 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:25.630393 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:25.635297 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:25.635273 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:26.448022 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:26.447981 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:17:26.523560 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:26.523508 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p5dtn" event={"ID":"0f534b12-a447-453d-aa10-9118ac1b6d53","Type":"ContainerStarted","Data":"637e8be282f316e9f0f814965d89940dc5f126fd9c47375d0718251229217403"} Sep 19 04:17:26.528488 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:26.528460 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:17:26.676556 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:26.676520 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7dfb47d4-246qx"] Sep 19 04:17:26.910753 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:26.910713 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:27.014090 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:27.014050 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:27.097244 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:27.097201 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:29.517049 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:29.517009 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:29.625428 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:29.625389 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:30.019002 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:30.018963 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:30.097260 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:30.097213 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:34.346538 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:34.346500 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:34.346938 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:34.346567 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:35.144575 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:35.144539 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:35.223076 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:35.223037 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:40.251952 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:40.251911 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:40.343239 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:40.343192 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:17:51.697430 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:51.697374 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b7dfb47d4-246qx" podUID="6b0153cc-54ae-477d-8d6c-cc8677a049e7" containerName="console" containerID="cri-o://a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4" gracePeriod=15 Sep 19 04:17:51.943193 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:51.943167 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:17:52.071939 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.071903 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-oauth-config\") pod \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " Sep 19 04:17:52.072131 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.071954 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-config\") pod \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " Sep 19 04:17:52.072131 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.071974 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-service-ca\") pod \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " Sep 19 04:17:52.072131 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.072000 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-oauth-serving-cert\") pod \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " Sep 19 04:17:52.072131 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.072056 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-serving-cert\") pod \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " Sep 19 04:17:52.072131 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.072092 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9248\" (UniqueName: \"kubernetes.io/projected/6b0153cc-54ae-477d-8d6c-cc8677a049e7-kube-api-access-x9248\") pod \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\" (UID: \"6b0153cc-54ae-477d-8d6c-cc8677a049e7\") " Sep 19 04:17:52.072439 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.072412 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-service-ca" (OuterVolumeSpecName: "service-ca") pod "6b0153cc-54ae-477d-8d6c-cc8677a049e7" (UID: "6b0153cc-54ae-477d-8d6c-cc8677a049e7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:17:52.072495 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.072420 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-config" (OuterVolumeSpecName: "console-config") pod "6b0153cc-54ae-477d-8d6c-cc8677a049e7" (UID: "6b0153cc-54ae-477d-8d6c-cc8677a049e7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:17:52.072495 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.072431 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6b0153cc-54ae-477d-8d6c-cc8677a049e7" (UID: "6b0153cc-54ae-477d-8d6c-cc8677a049e7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:17:52.074544 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.074515 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b0153cc-54ae-477d-8d6c-cc8677a049e7-kube-api-access-x9248" (OuterVolumeSpecName: "kube-api-access-x9248") pod "6b0153cc-54ae-477d-8d6c-cc8677a049e7" (UID: "6b0153cc-54ae-477d-8d6c-cc8677a049e7"). InnerVolumeSpecName "kube-api-access-x9248". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:17:52.074649 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.074597 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6b0153cc-54ae-477d-8d6c-cc8677a049e7" (UID: "6b0153cc-54ae-477d-8d6c-cc8677a049e7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:17:52.074649 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.074616 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6b0153cc-54ae-477d-8d6c-cc8677a049e7" (UID: "6b0153cc-54ae-477d-8d6c-cc8677a049e7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:17:52.173878 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.173837 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-serving-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:17:52.173878 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.173871 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9248\" (UniqueName: \"kubernetes.io/projected/6b0153cc-54ae-477d-8d6c-cc8677a049e7-kube-api-access-x9248\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:17:52.173878 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.173881 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-oauth-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:17:52.174107 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.173890 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-console-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:17:52.174107 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.173899 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-service-ca\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:17:52.174107 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.173907 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0153cc-54ae-477d-8d6c-cc8677a049e7-oauth-serving-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:17:52.638714 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.638683 2577 generic.go:356] "Generic (PLEG): container finished" podID="6b0153cc-54ae-477d-8d6c-cc8677a049e7" containerID="a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4" exitCode=0 Sep 19 04:17:52.638899 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.638726 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7dfb47d4-246qx" Sep 19 04:17:52.638899 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.638741 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7dfb47d4-246qx" event={"ID":"6b0153cc-54ae-477d-8d6c-cc8677a049e7","Type":"ContainerDied","Data":"a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4"} Sep 19 04:17:52.638899 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.638781 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7dfb47d4-246qx" event={"ID":"6b0153cc-54ae-477d-8d6c-cc8677a049e7","Type":"ContainerDied","Data":"84b7af8d8f2832a34d188cd91c1d992a9decfaa92f0eb2391ad9f0d8812acfc6"} Sep 19 04:17:52.638899 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.638801 2577 scope.go:117] "RemoveContainer" containerID="a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4" Sep 19 04:17:52.648433 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.648416 2577 scope.go:117] "RemoveContainer" containerID="a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4" Sep 19 04:17:52.648699 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:17:52.648680 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4\": container with ID starting with a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4 not found: ID does not exist" containerID="a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4" Sep 19 04:17:52.648803 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.648706 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4"} err="failed to get container status \"a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4\": rpc error: code = NotFound desc = could not find container \"a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4\": container with ID starting with a1201e7c6b27deeb7b8ee94d1a654bb0fec44ab0ead09f602197c44ab1d2f9a4 not found: ID does not exist" Sep 19 04:17:52.664335 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.664302 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7dfb47d4-246qx"] Sep 19 04:17:52.668469 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:52.668447 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b7dfb47d4-246qx"] Sep 19 04:17:53.893356 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:53.893323 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b0153cc-54ae-477d-8d6c-cc8677a049e7" path="/var/lib/kubelet/pods/6b0153cc-54ae-477d-8d6c-cc8677a049e7/volumes" Sep 19 04:17:54.351728 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:54.351698 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:17:54.355646 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:17:54.355623 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6dfd8fc58b-tq5h5" Sep 19 04:18:16.447756 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:16.447662 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:16.464210 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:16.464178 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:16.759896 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:16.759818 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:29.019971 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:29.019924 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:18:29.467529 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:29.467418 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:18:30.083840 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.083802 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 04:18:30.084240 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.084106 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="alertmanager" containerID="cri-o://9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b" gracePeriod=120 Sep 19 04:18:30.084240 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.084203 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="prom-label-proxy" containerID="cri-o://b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec" gracePeriod=120 Sep 19 04:18:30.084375 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.084236 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy-web" containerID="cri-o://739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441" gracePeriod=120 Sep 19 04:18:30.084375 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.084259 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="config-reloader" containerID="cri-o://c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3" gracePeriod=120 Sep 19 04:18:30.084375 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.084278 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy" containerID="cri-o://874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98" gracePeriod=120 Sep 19 04:18:30.084375 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.084207 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy-metric" containerID="cri-o://8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d" gracePeriod=120 Sep 19 04:18:30.803090 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.803051 2577 generic.go:356] "Generic (PLEG): container finished" podID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerID="b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec" exitCode=0 Sep 19 04:18:30.803090 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.803077 2577 generic.go:356] "Generic (PLEG): container finished" podID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerID="874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98" exitCode=0 Sep 19 04:18:30.803090 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.803083 2577 generic.go:356] "Generic (PLEG): container finished" podID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerID="c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3" exitCode=0 Sep 19 04:18:30.803090 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.803089 2577 generic.go:356] "Generic (PLEG): container finished" podID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerID="9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b" exitCode=0 Sep 19 04:18:30.803362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.803118 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerDied","Data":"b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec"} Sep 19 04:18:30.803362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.803198 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerDied","Data":"874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98"} Sep 19 04:18:30.803362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.803228 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerDied","Data":"c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3"} Sep 19 04:18:30.803362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:30.803252 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerDied","Data":"9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b"} Sep 19 04:18:31.331675 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.331649 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:18:31.442805 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.442716 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-cluster-tls-config\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.442805 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.442768 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-config-out\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443017 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.442807 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-main-tls\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443017 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.442843 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-web\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443017 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.442893 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-config-volume\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443017 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.442920 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-web-config\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443017 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.442948 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443017 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.443004 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-tls-assets\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443335 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.443040 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-metrics-client-ca\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443335 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.443077 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-main-db\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443335 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.443125 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5btw\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-kube-api-access-c5btw\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443335 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.443175 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.443335 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.443208 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-trusted-ca-bundle\") pod \"952c8259-3d05-45a7-aa71-3278a80f92f8\" (UID: \"952c8259-3d05-45a7-aa71-3278a80f92f8\") " Sep 19 04:18:31.445362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.445324 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-config-out" (OuterVolumeSpecName: "config-out") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 04:18:31.446785 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.446364 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 04:18:31.446785 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.446519 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-config-volume" (OuterVolumeSpecName: "config-volume") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:31.446785 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.446595 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:18:31.446785 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.446635 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:31.447047 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.446905 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:18:31.449261 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.447738 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:18:31.453166 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.453118 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:31.453482 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.453456 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:31.454253 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.454135 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:31.454345 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.454308 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-kube-api-access-c5btw" (OuterVolumeSpecName: "kube-api-access-c5btw") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "kube-api-access-c5btw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:18:31.457314 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.457289 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:31.463801 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.463779 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-web-config" (OuterVolumeSpecName: "web-config") pod "952c8259-3d05-45a7-aa71-3278a80f92f8" (UID: "952c8259-3d05-45a7-aa71-3278a80f92f8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:31.545248 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545204 2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-cluster-tls-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545248 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545235 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-config-out\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545248 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545247 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-main-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545258 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545268 2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-config-volume\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545293 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-web-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545302 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545311 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-tls-assets\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545319 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-metrics-client-ca\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545329 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-main-db\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545339 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5btw\" (UniqueName: \"kubernetes.io/projected/952c8259-3d05-45a7-aa71-3278a80f92f8-kube-api-access-c5btw\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545347 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/952c8259-3d05-45a7-aa71-3278a80f92f8-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.545480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.545356 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952c8259-3d05-45a7-aa71-3278a80f92f8-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:31.810680 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.810648 2577 generic.go:356] "Generic (PLEG): container finished" podID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerID="8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d" exitCode=0 Sep 19 04:18:31.810680 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.810674 2577 generic.go:356] "Generic (PLEG): container finished" podID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerID="739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441" exitCode=0 Sep 19 04:18:31.810923 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.810723 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerDied","Data":"8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d"} Sep 19 04:18:31.810923 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.810797 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 04:18:31.810923 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.810829 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerDied","Data":"739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441"} Sep 19 04:18:31.810923 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.810871 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"952c8259-3d05-45a7-aa71-3278a80f92f8","Type":"ContainerDied","Data":"24d35c2f67b4274929f65962c8b3c65250f47ab60a3fbafc668ed39e2ed99ed6"} Sep 19 04:18:31.810923 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.810893 2577 scope.go:117] "RemoveContainer" containerID="b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec" Sep 19 04:18:31.819047 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.819029 2577 scope.go:117] "RemoveContainer" containerID="8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d" Sep 19 04:18:31.826054 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.826020 2577 scope.go:117] "RemoveContainer" containerID="874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98" Sep 19 04:18:31.832946 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.832927 2577 scope.go:117] "RemoveContainer" containerID="739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441" Sep 19 04:18:31.839638 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.839620 2577 scope.go:117] "RemoveContainer" containerID="c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3" Sep 19 04:18:31.847234 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.847202 2577 scope.go:117] "RemoveContainer" containerID="9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b" Sep 19 04:18:31.847582 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.847562 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 04:18:31.854069 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.854043 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 04:18:31.855595 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.855577 2577 scope.go:117] "RemoveContainer" containerID="87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d" Sep 19 04:18:31.862614 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.862598 2577 scope.go:117] "RemoveContainer" containerID="b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec" Sep 19 04:18:31.862886 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:31.862868 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec\": container with ID starting with b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec not found: ID does not exist" containerID="b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec" Sep 19 04:18:31.862951 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.862895 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec"} err="failed to get container status \"b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec\": rpc error: code = NotFound desc = could not find container \"b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec\": container with ID starting with b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec not found: ID does not exist" Sep 19 04:18:31.862951 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.862914 2577 scope.go:117] "RemoveContainer" containerID="8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d" Sep 19 04:18:31.863188 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:31.863141 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d\": container with ID starting with 8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d not found: ID does not exist" containerID="8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d" Sep 19 04:18:31.863273 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.863195 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d"} err="failed to get container status \"8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d\": rpc error: code = NotFound desc = could not find container \"8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d\": container with ID starting with 8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d not found: ID does not exist" Sep 19 04:18:31.863273 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.863212 2577 scope.go:117] "RemoveContainer" containerID="874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98" Sep 19 04:18:31.863439 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:31.863422 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98\": container with ID starting with 874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98 not found: ID does not exist" containerID="874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98" Sep 19 04:18:31.863477 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.863444 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98"} err="failed to get container status \"874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98\": rpc error: code = NotFound desc = could not find container \"874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98\": container with ID starting with 874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98 not found: ID does not exist" Sep 19 04:18:31.863477 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.863458 2577 scope.go:117] "RemoveContainer" containerID="739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441" Sep 19 04:18:31.863709 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:31.863692 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441\": container with ID starting with 739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441 not found: ID does not exist" containerID="739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441" Sep 19 04:18:31.863764 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.863712 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441"} err="failed to get container status \"739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441\": rpc error: code = NotFound desc = could not find container \"739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441\": container with ID starting with 739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441 not found: ID does not exist" Sep 19 04:18:31.863764 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.863725 2577 scope.go:117] "RemoveContainer" containerID="c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3" Sep 19 04:18:31.863959 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:31.863943 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3\": container with ID starting with c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3 not found: ID does not exist" containerID="c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3" Sep 19 04:18:31.863993 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.863965 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3"} err="failed to get container status \"c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3\": rpc error: code = NotFound desc = could not find container \"c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3\": container with ID starting with c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3 not found: ID does not exist" Sep 19 04:18:31.863993 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.863981 2577 scope.go:117] "RemoveContainer" containerID="9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b" Sep 19 04:18:31.864289 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:31.864271 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b\": container with ID starting with 9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b not found: ID does not exist" containerID="9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b" Sep 19 04:18:31.864350 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.864292 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b"} err="failed to get container status \"9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b\": rpc error: code = NotFound desc = could not find container \"9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b\": container with ID starting with 9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b not found: ID does not exist" Sep 19 04:18:31.864350 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.864305 2577 scope.go:117] "RemoveContainer" containerID="87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d" Sep 19 04:18:31.864509 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:31.864494 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d\": container with ID starting with 87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d not found: ID does not exist" containerID="87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d" Sep 19 04:18:31.864548 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.864512 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d"} err="failed to get container status \"87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d\": rpc error: code = NotFound desc = could not find container \"87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d\": container with ID starting with 87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d not found: ID does not exist" Sep 19 04:18:31.864548 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.864526 2577 scope.go:117] "RemoveContainer" containerID="b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec" Sep 19 04:18:31.864744 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.864729 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec"} err="failed to get container status \"b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec\": rpc error: code = NotFound desc = could not find container \"b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec\": container with ID starting with b16b5150ad63b92416a046b4c2e8e407dd17f9453004fa5627f2035b2e7731ec not found: ID does not exist" Sep 19 04:18:31.864787 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.864745 2577 scope.go:117] "RemoveContainer" containerID="8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d" Sep 19 04:18:31.864961 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.864945 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d"} err="failed to get container status \"8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d\": rpc error: code = NotFound desc = could not find container \"8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d\": container with ID starting with 8efafa5027be1fe8c4ad04f370828f0bbd93ec0ce02c8235f7cfe72e9058c79d not found: ID does not exist" Sep 19 04:18:31.864961 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.864960 2577 scope.go:117] "RemoveContainer" containerID="874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98" Sep 19 04:18:31.865189 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.865173 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98"} err="failed to get container status \"874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98\": rpc error: code = NotFound desc = could not find container \"874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98\": container with ID starting with 874f937f43b7a2c63c55e7a352e8988e6294b22b8c4d9e7ef6eb64c3fa5b1a98 not found: ID does not exist" Sep 19 04:18:31.865242 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.865189 2577 scope.go:117] "RemoveContainer" containerID="739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441" Sep 19 04:18:31.865447 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.865427 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441"} err="failed to get container status \"739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441\": rpc error: code = NotFound desc = could not find container \"739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441\": container with ID starting with 739a95c058ca50bd10a689ca630a4a276ff56e54269fce08f027f72bfeb1a441 not found: ID does not exist" Sep 19 04:18:31.865499 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.865448 2577 scope.go:117] "RemoveContainer" containerID="c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3" Sep 19 04:18:31.865705 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.865688 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3"} err="failed to get container status \"c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3\": rpc error: code = NotFound desc = could not find container \"c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3\": container with ID starting with c598927e35285ad8171c29367f09f3738e425b8214eb64e31102f9c313938de3 not found: ID does not exist" Sep 19 04:18:31.865762 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.865705 2577 scope.go:117] "RemoveContainer" containerID="9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b" Sep 19 04:18:31.865944 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.865928 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b"} err="failed to get container status \"9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b\": rpc error: code = NotFound desc = could not find container \"9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b\": container with ID starting with 9c411ea7430e6bb703dcc9db2b940161a65894d109e9db91a142ed088a96d69b not found: ID does not exist" Sep 19 04:18:31.865944 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.865944 2577 scope.go:117] "RemoveContainer" containerID="87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d" Sep 19 04:18:31.866200 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.866181 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d"} err="failed to get container status \"87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d\": rpc error: code = NotFound desc = could not find container \"87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d\": container with ID starting with 87e8f8e7fb0c4a8d009b2932e17fe3ae79a20af640290d059ef01c96294c071d not found: ID does not exist" Sep 19 04:18:31.893545 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:31.893518 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" path="/var/lib/kubelet/pods/952c8259-3d05-45a7-aa71-3278a80f92f8/volumes" Sep 19 04:18:34.520675 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.520639 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 04:18:34.521566 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.521531 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="prometheus" containerID="cri-o://90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" gracePeriod=600 Sep 19 04:18:34.522047 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.522022 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" gracePeriod=600 Sep 19 04:18:34.522281 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.522261 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy" containerID="cri-o://e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" gracePeriod=600 Sep 19 04:18:34.522456 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.522437 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy-web" containerID="cri-o://fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" gracePeriod=600 Sep 19 04:18:34.522639 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.522619 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="thanos-sidecar" containerID="cri-o://2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" gracePeriod=600 Sep 19 04:18:34.522771 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.522649 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="config-reloader" containerID="cri-o://4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" gracePeriod=600 Sep 19 04:18:34.774080 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.774020 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:34.837681 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.837644 2577 generic.go:356] "Generic (PLEG): container finished" podID="796ea408-c763-48f1-80ca-8e21977b17b7" containerID="5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" exitCode=0 Sep 19 04:18:34.837865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.837721 2577 generic.go:356] "Generic (PLEG): container finished" podID="796ea408-c763-48f1-80ca-8e21977b17b7" containerID="e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" exitCode=0 Sep 19 04:18:34.837865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.837737 2577 generic.go:356] "Generic (PLEG): container finished" podID="796ea408-c763-48f1-80ca-8e21977b17b7" containerID="fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" exitCode=0 Sep 19 04:18:34.837865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.837747 2577 generic.go:356] "Generic (PLEG): container finished" podID="796ea408-c763-48f1-80ca-8e21977b17b7" containerID="2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" exitCode=0 Sep 19 04:18:34.837865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.837757 2577 generic.go:356] "Generic (PLEG): container finished" podID="796ea408-c763-48f1-80ca-8e21977b17b7" containerID="4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" exitCode=0 Sep 19 04:18:34.837865 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.837765 2577 generic.go:356] "Generic (PLEG): container finished" podID="796ea408-c763-48f1-80ca-8e21977b17b7" containerID="90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" exitCode=0 Sep 19 04:18:34.838122 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.837896 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerDied","Data":"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5"} Sep 19 04:18:34.838122 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.838008 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerDied","Data":"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec"} Sep 19 04:18:34.838122 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.838030 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:34.838122 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.838054 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerDied","Data":"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3"} Sep 19 04:18:34.838122 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.838097 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerDied","Data":"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269"} Sep 19 04:18:34.838392 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.838139 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerDied","Data":"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c"} Sep 19 04:18:34.838392 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.838210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerDied","Data":"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e"} Sep 19 04:18:34.838392 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.838251 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"796ea408-c763-48f1-80ca-8e21977b17b7","Type":"ContainerDied","Data":"766cf9fb5630b781da7c6ca4c6bdd2b695bc048c133a13108698caca4f23f16c"} Sep 19 04:18:34.838392 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.838298 2577 scope.go:117] "RemoveContainer" containerID="5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" Sep 19 04:18:34.848686 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.848649 2577 scope.go:117] "RemoveContainer" containerID="e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" Sep 19 04:18:34.858909 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.857980 2577 scope.go:117] "RemoveContainer" containerID="fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" Sep 19 04:18:34.866828 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.866809 2577 scope.go:117] "RemoveContainer" containerID="2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" Sep 19 04:18:34.872175 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872131 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-metrics-client-certs\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872273 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872219 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-2\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872273 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872245 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-grpc-tls\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872367 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872283 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-config-out\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872367 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872323 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872469 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872392 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872469 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872420 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-tls\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872469 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872464 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-serving-certs-ca-bundle\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872612 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872493 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-thanos-prometheus-http-client-file\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872612 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872526 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-1\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872612 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872551 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-web-config\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872612 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872575 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-metrics-client-ca\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872612 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872596 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-db\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872612 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872612 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-tls-assets\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872922 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872630 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tngc5\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-kube-api-access-tngc5\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872922 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872653 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:18:34.872922 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872660 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-0\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872922 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872722 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-trusted-ca-bundle\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872922 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872751 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-config\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872922 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872781 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-kubelet-serving-ca-bundle\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.872922 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.872811 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-kube-rbac-proxy\") pod \"796ea408-c763-48f1-80ca-8e21977b17b7\" (UID: \"796ea408-c763-48f1-80ca-8e21977b17b7\") " Sep 19 04:18:34.873476 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.873167 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.874450 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.874423 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:18:34.874544 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.874460 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:18:34.876137 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.876013 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:18:34.876493 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.876354 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:18:34.876493 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.876434 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 04:18:34.876632 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.876497 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:34.876984 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.876870 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:18:34.877108 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.877039 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:18:34.877964 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.877913 2577 scope.go:117] "RemoveContainer" containerID="4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" Sep 19 04:18:34.878332 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.878213 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:18:34.878538 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.878516 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:34.878869 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.878842 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:34.878980 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.878863 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:34.879044 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.878979 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:34.879044 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.878995 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:34.879189 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.879057 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:34.879575 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.879543 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-config" (OuterVolumeSpecName: "config") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:34.879654 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.879585 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-kube-api-access-tngc5" (OuterVolumeSpecName: "kube-api-access-tngc5") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "kube-api-access-tngc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:18:34.881634 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.881497 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-config-out" (OuterVolumeSpecName: "config-out") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 04:18:34.888195 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.888135 2577 scope.go:117] "RemoveContainer" containerID="90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" Sep 19 04:18:34.891060 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.891038 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-web-config" (OuterVolumeSpecName: "web-config") pod "796ea408-c763-48f1-80ca-8e21977b17b7" (UID: "796ea408-c763-48f1-80ca-8e21977b17b7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:18:34.895292 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.895269 2577 scope.go:117] "RemoveContainer" containerID="6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d" Sep 19 04:18:34.902371 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.902350 2577 scope.go:117] "RemoveContainer" containerID="5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" Sep 19 04:18:34.902674 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:34.902655 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": container with ID starting with 5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5 not found: ID does not exist" containerID="5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" Sep 19 04:18:34.902747 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.902685 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5"} err="failed to get container status \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": rpc error: code = NotFound desc = could not find container \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": container with ID starting with 5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5 not found: ID does not exist" Sep 19 04:18:34.902747 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.902704 2577 scope.go:117] "RemoveContainer" containerID="e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" Sep 19 04:18:34.902994 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:34.902978 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": container with ID starting with e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec not found: ID does not exist" containerID="e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" Sep 19 04:18:34.903060 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.902999 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec"} err="failed to get container status \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": rpc error: code = NotFound desc = could not find container \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": container with ID starting with e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec not found: ID does not exist" Sep 19 04:18:34.903060 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.903013 2577 scope.go:117] "RemoveContainer" containerID="fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" Sep 19 04:18:34.903302 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:34.903286 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": container with ID starting with fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3 not found: ID does not exist" containerID="fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" Sep 19 04:18:34.903374 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.903307 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3"} err="failed to get container status \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": rpc error: code = NotFound desc = could not find container \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": container with ID starting with fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3 not found: ID does not exist" Sep 19 04:18:34.903374 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.903320 2577 scope.go:117] "RemoveContainer" containerID="2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" Sep 19 04:18:34.903583 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:34.903555 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": container with ID starting with 2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269 not found: ID does not exist" containerID="2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" Sep 19 04:18:34.903629 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.903587 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269"} err="failed to get container status \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": rpc error: code = NotFound desc = could not find container \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": container with ID starting with 2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269 not found: ID does not exist" Sep 19 04:18:34.903629 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.903603 2577 scope.go:117] "RemoveContainer" containerID="4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" Sep 19 04:18:34.903853 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:34.903832 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": container with ID starting with 4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c not found: ID does not exist" containerID="4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" Sep 19 04:18:34.903933 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.903860 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c"} err="failed to get container status \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": rpc error: code = NotFound desc = could not find container \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": container with ID starting with 4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c not found: ID does not exist" Sep 19 04:18:34.903933 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.903879 2577 scope.go:117] "RemoveContainer" containerID="90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" Sep 19 04:18:34.904176 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:34.904143 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": container with ID starting with 90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e not found: ID does not exist" containerID="90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" Sep 19 04:18:34.904231 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.904179 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e"} err="failed to get container status \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": rpc error: code = NotFound desc = could not find container \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": container with ID starting with 90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e not found: ID does not exist" Sep 19 04:18:34.904231 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.904192 2577 scope.go:117] "RemoveContainer" containerID="6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d" Sep 19 04:18:34.904437 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:18:34.904418 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": container with ID starting with 6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d not found: ID does not exist" containerID="6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d" Sep 19 04:18:34.904504 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.904443 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d"} err="failed to get container status \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": rpc error: code = NotFound desc = could not find container \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": container with ID starting with 6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d not found: ID does not exist" Sep 19 04:18:34.904504 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.904464 2577 scope.go:117] "RemoveContainer" containerID="5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" Sep 19 04:18:34.904724 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.904705 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5"} err="failed to get container status \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": rpc error: code = NotFound desc = could not find container \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": container with ID starting with 5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5 not found: ID does not exist" Sep 19 04:18:34.904800 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.904726 2577 scope.go:117] "RemoveContainer" containerID="e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" Sep 19 04:18:34.904966 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.904947 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec"} err="failed to get container status \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": rpc error: code = NotFound desc = could not find container \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": container with ID starting with e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec not found: ID does not exist" Sep 19 04:18:34.905034 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.904966 2577 scope.go:117] "RemoveContainer" containerID="fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" Sep 19 04:18:34.905278 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.905261 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3"} err="failed to get container status \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": rpc error: code = NotFound desc = could not find container \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": container with ID starting with fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3 not found: ID does not exist" Sep 19 04:18:34.905337 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.905280 2577 scope.go:117] "RemoveContainer" containerID="2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" Sep 19 04:18:34.905541 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.905522 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269"} err="failed to get container status \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": rpc error: code = NotFound desc = could not find container \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": container with ID starting with 2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269 not found: ID does not exist" Sep 19 04:18:34.905615 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.905542 2577 scope.go:117] "RemoveContainer" containerID="4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" Sep 19 04:18:34.905815 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.905799 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c"} err="failed to get container status \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": rpc error: code = NotFound desc = could not find container \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": container with ID starting with 4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c not found: ID does not exist" Sep 19 04:18:34.905877 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.905817 2577 scope.go:117] "RemoveContainer" containerID="90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" Sep 19 04:18:34.906056 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.906038 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e"} err="failed to get container status \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": rpc error: code = NotFound desc = could not find container \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": container with ID starting with 90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e not found: ID does not exist" Sep 19 04:18:34.906056 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.906054 2577 scope.go:117] "RemoveContainer" containerID="6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d" Sep 19 04:18:34.906337 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.906321 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d"} err="failed to get container status \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": rpc error: code = NotFound desc = could not find container \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": container with ID starting with 6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d not found: ID does not exist" Sep 19 04:18:34.906391 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.906339 2577 scope.go:117] "RemoveContainer" containerID="5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" Sep 19 04:18:34.906578 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.906558 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5"} err="failed to get container status \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": rpc error: code = NotFound desc = could not find container \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": container with ID starting with 5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5 not found: ID does not exist" Sep 19 04:18:34.906641 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.906580 2577 scope.go:117] "RemoveContainer" containerID="e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" Sep 19 04:18:34.906830 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.906811 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec"} err="failed to get container status \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": rpc error: code = NotFound desc = could not find container \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": container with ID starting with e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec not found: ID does not exist" Sep 19 04:18:34.906882 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.906831 2577 scope.go:117] "RemoveContainer" containerID="fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" Sep 19 04:18:34.907041 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907023 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3"} err="failed to get container status \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": rpc error: code = NotFound desc = could not find container \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": container with ID starting with fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3 not found: ID does not exist" Sep 19 04:18:34.907091 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907042 2577 scope.go:117] "RemoveContainer" containerID="2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" Sep 19 04:18:34.907298 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907279 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269"} err="failed to get container status \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": rpc error: code = NotFound desc = could not find container \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": container with ID starting with 2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269 not found: ID does not exist" Sep 19 04:18:34.907378 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907299 2577 scope.go:117] "RemoveContainer" containerID="4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" Sep 19 04:18:34.907555 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907537 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c"} err="failed to get container status \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": rpc error: code = NotFound desc = could not find container \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": container with ID starting with 4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c not found: ID does not exist" Sep 19 04:18:34.907599 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907555 2577 scope.go:117] "RemoveContainer" containerID="90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" Sep 19 04:18:34.907767 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907744 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e"} err="failed to get container status \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": rpc error: code = NotFound desc = could not find container \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": container with ID starting with 90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e not found: ID does not exist" Sep 19 04:18:34.907823 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907767 2577 scope.go:117] "RemoveContainer" containerID="6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d" Sep 19 04:18:34.907966 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907951 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d"} err="failed to get container status \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": rpc error: code = NotFound desc = could not find container \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": container with ID starting with 6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d not found: ID does not exist" Sep 19 04:18:34.908006 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.907966 2577 scope.go:117] "RemoveContainer" containerID="5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" Sep 19 04:18:34.908188 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.908145 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5"} err="failed to get container status \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": rpc error: code = NotFound desc = could not find container \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": container with ID starting with 5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5 not found: ID does not exist" Sep 19 04:18:34.908268 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.908190 2577 scope.go:117] "RemoveContainer" containerID="e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" Sep 19 04:18:34.908403 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.908387 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec"} err="failed to get container status \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": rpc error: code = NotFound desc = could not find container \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": container with ID starting with e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec not found: ID does not exist" Sep 19 04:18:34.908475 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.908405 2577 scope.go:117] "RemoveContainer" containerID="fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" Sep 19 04:18:34.908614 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.908595 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3"} err="failed to get container status \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": rpc error: code = NotFound desc = could not find container \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": container with ID starting with fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3 not found: ID does not exist" Sep 19 04:18:34.908614 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.908611 2577 scope.go:117] "RemoveContainer" containerID="2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" Sep 19 04:18:34.908842 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.908826 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269"} err="failed to get container status \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": rpc error: code = NotFound desc = could not find container \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": container with ID starting with 2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269 not found: ID does not exist" Sep 19 04:18:34.908842 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.908841 2577 scope.go:117] "RemoveContainer" containerID="4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" Sep 19 04:18:34.909076 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.909059 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c"} err="failed to get container status \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": rpc error: code = NotFound desc = could not find container \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": container with ID starting with 4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c not found: ID does not exist" Sep 19 04:18:34.909119 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.909076 2577 scope.go:117] "RemoveContainer" containerID="90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" Sep 19 04:18:34.909335 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.909315 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e"} err="failed to get container status \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": rpc error: code = NotFound desc = could not find container \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": container with ID starting with 90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e not found: ID does not exist" Sep 19 04:18:34.909406 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.909336 2577 scope.go:117] "RemoveContainer" containerID="6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d" Sep 19 04:18:34.909584 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.909566 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d"} err="failed to get container status \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": rpc error: code = NotFound desc = could not find container \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": container with ID starting with 6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d not found: ID does not exist" Sep 19 04:18:34.909675 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.909586 2577 scope.go:117] "RemoveContainer" containerID="5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" Sep 19 04:18:34.909821 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.909800 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5"} err="failed to get container status \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": rpc error: code = NotFound desc = could not find container \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": container with ID starting with 5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5 not found: ID does not exist" Sep 19 04:18:34.909887 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.909821 2577 scope.go:117] "RemoveContainer" containerID="e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" Sep 19 04:18:34.910040 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.910022 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec"} err="failed to get container status \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": rpc error: code = NotFound desc = could not find container \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": container with ID starting with e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec not found: ID does not exist" Sep 19 04:18:34.910113 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.910042 2577 scope.go:117] "RemoveContainer" containerID="fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" Sep 19 04:18:34.910336 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.910318 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3"} err="failed to get container status \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": rpc error: code = NotFound desc = could not find container \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": container with ID starting with fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3 not found: ID does not exist" Sep 19 04:18:34.910407 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.910337 2577 scope.go:117] "RemoveContainer" containerID="2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" Sep 19 04:18:34.910573 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.910555 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269"} err="failed to get container status \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": rpc error: code = NotFound desc = could not find container \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": container with ID starting with 2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269 not found: ID does not exist" Sep 19 04:18:34.910610 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.910573 2577 scope.go:117] "RemoveContainer" containerID="4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" Sep 19 04:18:34.910787 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.910771 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c"} err="failed to get container status \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": rpc error: code = NotFound desc = could not find container \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": container with ID starting with 4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c not found: ID does not exist" Sep 19 04:18:34.910825 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.910787 2577 scope.go:117] "RemoveContainer" containerID="90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" Sep 19 04:18:34.911010 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.910994 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e"} err="failed to get container status \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": rpc error: code = NotFound desc = could not find container \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": container with ID starting with 90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e not found: ID does not exist" Sep 19 04:18:34.911048 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.911010 2577 scope.go:117] "RemoveContainer" containerID="6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d" Sep 19 04:18:34.911250 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.911229 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d"} err="failed to get container status \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": rpc error: code = NotFound desc = could not find container \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": container with ID starting with 6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d not found: ID does not exist" Sep 19 04:18:34.911296 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.911253 2577 scope.go:117] "RemoveContainer" containerID="5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5" Sep 19 04:18:34.911509 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.911489 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5"} err="failed to get container status \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": rpc error: code = NotFound desc = could not find container \"5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5\": container with ID starting with 5a6e52899efe52b92541d837c70afd73fd99be71dcf7f32e2cacb0919e4bc0e5 not found: ID does not exist" Sep 19 04:18:34.911509 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.911509 2577 scope.go:117] "RemoveContainer" containerID="e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec" Sep 19 04:18:34.911764 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.911744 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec"} err="failed to get container status \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": rpc error: code = NotFound desc = could not find container \"e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec\": container with ID starting with e7fbc2bcb843fde1d176664da7f4f321b55babd88c5e2a1480902087a63738ec not found: ID does not exist" Sep 19 04:18:34.911764 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.911762 2577 scope.go:117] "RemoveContainer" containerID="fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3" Sep 19 04:18:34.912025 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.912005 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3"} err="failed to get container status \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": rpc error: code = NotFound desc = could not find container \"fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3\": container with ID starting with fd56deb145e9e3b9db1c7e17db3566ccfa3cedfbaccec321c8b76b540fdb7ff3 not found: ID does not exist" Sep 19 04:18:34.912088 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.912026 2577 scope.go:117] "RemoveContainer" containerID="2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269" Sep 19 04:18:34.912297 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.912279 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269"} err="failed to get container status \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": rpc error: code = NotFound desc = could not find container \"2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269\": container with ID starting with 2ac9941cdddae5bbdcf9ab68c8b65d56144cd0f522fb150ebe3a2c506eb61269 not found: ID does not exist" Sep 19 04:18:34.912376 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.912298 2577 scope.go:117] "RemoveContainer" containerID="4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c" Sep 19 04:18:34.912614 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.912591 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c"} err="failed to get container status \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": rpc error: code = NotFound desc = could not find container \"4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c\": container with ID starting with 4f630e320ab3d8e057981a192550d72797938157d15758a5b571d46491279c1c not found: ID does not exist" Sep 19 04:18:34.912696 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.912615 2577 scope.go:117] "RemoveContainer" containerID="90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e" Sep 19 04:18:34.912873 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.912856 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e"} err="failed to get container status \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": rpc error: code = NotFound desc = could not find container \"90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e\": container with ID starting with 90b36f607df8972f6375fddd9e3d9f39fb550bfec45545310e76e078b03d723e not found: ID does not exist" Sep 19 04:18:34.912940 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.912876 2577 scope.go:117] "RemoveContainer" containerID="6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d" Sep 19 04:18:34.913128 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.913109 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d"} err="failed to get container status \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": rpc error: code = NotFound desc = could not find container \"6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d\": container with ID starting with 6544dcb1261e2a6899f8128f549d9c17a95d9417b2b40f3d09725605d27e154d not found: ID does not exist" Sep 19 04:18:34.973724 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973698 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-db\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973724 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973729 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-tls-assets\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973741 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tngc5\" (UniqueName: \"kubernetes.io/projected/796ea408-c763-48f1-80ca-8e21977b17b7-kube-api-access-tngc5\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973756 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973767 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973777 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973786 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973795 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-kube-rbac-proxy\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973805 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-metrics-client-certs\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973813 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-grpc-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973822 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/796ea408-c763-48f1-80ca-8e21977b17b7-config-out\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973836 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973848 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973856 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973865 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973873 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973885 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973892 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/796ea408-c763-48f1-80ca-8e21977b17b7-web-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:34.973897 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:34.973901 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/796ea408-c763-48f1-80ca-8e21977b17b7-configmap-metrics-client-ca\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:18:35.166443 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.166414 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 04:18:35.172776 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.172742 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 04:18:35.220990 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.220956 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 04:18:35.221255 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221242 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="prom-label-proxy" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221256 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="prom-label-proxy" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221267 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="init-config-reloader" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221276 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="init-config-reloader" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221287 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="prometheus" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221295 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="prometheus" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221301 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy-web" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221307 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy-web" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221320 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b0153cc-54ae-477d-8d6c-cc8677a049e7" containerName="console" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221328 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b0153cc-54ae-477d-8d6c-cc8677a049e7" containerName="console" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221337 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy-metric" Sep 19 04:18:35.221341 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221343 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy-metric" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221354 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="init-config-reloader" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221363 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="init-config-reloader" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221374 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="config-reloader" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221379 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="config-reloader" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221385 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221390 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221404 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="alertmanager" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221410 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="alertmanager" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221418 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy-web" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221423 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy-web" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221431 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="thanos-sidecar" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221440 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="thanos-sidecar" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221454 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="config-reloader" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221461 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="config-reloader" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221471 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221478 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221487 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy-thanos" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221495 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy-thanos" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221577 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy-web" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221590 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="prom-label-proxy" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221600 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="config-reloader" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221609 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="alertmanager" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221618 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy-thanos" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221626 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b0153cc-54ae-477d-8d6c-cc8677a049e7" containerName="console" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221632 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="thanos-sidecar" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221637 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy-web" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221643 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="kube-rbac-proxy" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221648 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221653 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" containerName="prometheus" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221658 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="config-reloader" Sep 19 04:18:35.221841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.221662 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="952c8259-3d05-45a7-aa71-3278a80f92f8" containerName="kube-rbac-proxy-metric" Sep 19 04:18:35.225880 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.225860 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.232587 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.232561 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 19 04:18:35.232827 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.232810 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 19 04:18:35.233512 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.233494 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 19 04:18:35.233604 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.233497 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 19 04:18:35.233967 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.233951 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 19 04:18:35.234096 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.234080 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 19 04:18:35.234182 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.234096 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 19 04:18:35.234506 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.234490 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 19 04:18:35.234548 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.234504 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e6dr04e4804kb\"" Sep 19 04:18:35.234582 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.234559 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 19 04:18:35.234907 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.234891 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 19 04:18:35.234974 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.234916 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-x2fn9\"" Sep 19 04:18:35.235081 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.234970 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 19 04:18:35.242091 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.242073 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 19 04:18:35.242869 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.242851 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 19 04:18:35.245019 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.245003 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 19 04:18:35.253926 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.253905 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 04:18:35.378063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378063 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-config\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378321 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378084 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378321 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378194 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378321 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378225 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378321 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378243 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378321 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378260 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378321 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378281 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378321 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378300 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxvm6\" (UniqueName: \"kubernetes.io/projected/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-kube-api-access-bxvm6\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378365 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378393 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378429 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378456 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378488 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378504 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378552 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378547 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378829 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378564 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.378829 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.378581 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479494 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479403 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479494 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479461 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479686 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479492 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479686 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479516 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479686 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479686 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479567 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479686 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479591 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479931 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479931 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479802 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-config\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479931 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479832 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479931 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479886 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.479931 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480223 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479936 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480223 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479950 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480223 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.479959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480223 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.480002 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480223 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.480036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxvm6\" (UniqueName: \"kubernetes.io/projected/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-kube-api-access-bxvm6\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480223 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.480075 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480223 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.480110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480223 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.480135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480223 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.480205 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480826 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.480274 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480826 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.480290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.480826 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.480804 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.482983 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.482883 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.482983 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.482883 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.483192 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.483035 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.483192 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.483069 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.483417 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.483393 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.483680 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.483632 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.484015 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.483993 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.484176 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.484013 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.484306 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.484086 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.484306 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.484223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.484468 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.484436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.485664 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.485640 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-config\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.485984 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.485965 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.486062 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.485972 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.486237 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.486219 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.494333 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.494315 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxvm6\" (UniqueName: \"kubernetes.io/projected/be47a7bd-f90d-4a04-9be2-e84845dd0bbf-kube-api-access-bxvm6\") pod \"prometheus-k8s-0\" (UID: \"be47a7bd-f90d-4a04-9be2-e84845dd0bbf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.534796 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.534769 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:35.862664 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:18:35.862633 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe47a7bd_f90d_4a04_9be2_e84845dd0bbf.slice/crio-86716b0e42babd47070408eeaa3ffeb2fcf9a625b6d06525b5e749c5b736d587 WatchSource:0}: Error finding container 86716b0e42babd47070408eeaa3ffeb2fcf9a625b6d06525b5e749c5b736d587: Status 404 returned error can't find the container with id 86716b0e42babd47070408eeaa3ffeb2fcf9a625b6d06525b5e749c5b736d587 Sep 19 04:18:35.866600 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.866575 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 04:18:35.893692 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:35.893666 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="796ea408-c763-48f1-80ca-8e21977b17b7" path="/var/lib/kubelet/pods/796ea408-c763-48f1-80ca-8e21977b17b7/volumes" Sep 19 04:18:36.846127 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:36.846091 2577 generic.go:356] "Generic (PLEG): container finished" podID="be47a7bd-f90d-4a04-9be2-e84845dd0bbf" containerID="1530e9de3588556397ee6866282ec3f156c1d7beaec814f4674587ab47bc0961" exitCode=0 Sep 19 04:18:36.846501 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:36.846184 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be47a7bd-f90d-4a04-9be2-e84845dd0bbf","Type":"ContainerDied","Data":"1530e9de3588556397ee6866282ec3f156c1d7beaec814f4674587ab47bc0961"} Sep 19 04:18:36.846501 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:36.846276 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be47a7bd-f90d-4a04-9be2-e84845dd0bbf","Type":"ContainerStarted","Data":"86716b0e42babd47070408eeaa3ffeb2fcf9a625b6d06525b5e749c5b736d587"} Sep 19 04:18:37.853982 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:37.853942 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be47a7bd-f90d-4a04-9be2-e84845dd0bbf","Type":"ContainerStarted","Data":"59bf4a38e8265f5eb5180a907a38ba41b3e648e7aa873b19ea843725174e82b2"} Sep 19 04:18:37.854488 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:37.854021 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be47a7bd-f90d-4a04-9be2-e84845dd0bbf","Type":"ContainerStarted","Data":"b4a5f9e391f0389e69b6a851bf29f73edf62040f583e53f3d8286a7a96c9f821"} Sep 19 04:18:37.854488 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:37.854066 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be47a7bd-f90d-4a04-9be2-e84845dd0bbf","Type":"ContainerStarted","Data":"3d8cbbe6af1fd25f5aa68307ddbe8bf98604c7a768fbc06622d7d9a822ecbfb2"} Sep 19 04:18:37.854488 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:37.854107 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be47a7bd-f90d-4a04-9be2-e84845dd0bbf","Type":"ContainerStarted","Data":"acc7dc6cb561e4dc822722cd691027d39672bd755802f25f981295e82afc81b4"} Sep 19 04:18:37.854488 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:37.854175 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be47a7bd-f90d-4a04-9be2-e84845dd0bbf","Type":"ContainerStarted","Data":"2d4caa90bc5e2b4375399278c2280c4edae1ba60d2e8e6bb3d218040f4c53904"} Sep 19 04:18:37.854488 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:37.854223 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be47a7bd-f90d-4a04-9be2-e84845dd0bbf","Type":"ContainerStarted","Data":"c66121a394c6758ce4a523f254148e2203102692fd899974dc9008cb33847b8c"} Sep 19 04:18:37.931092 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:37.931043 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.908310925 podStartE2EDuration="2.931025662s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.714737ms" isStatefulPod=false podCreationTimestamp="2026-09-19 04:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:18:37.930241021 +0000 UTC m=+184.251339194" watchObservedRunningTime="2026-09-19 04:18:37.931025662 +0000 UTC m=+184.252123807" Sep 19 04:18:38.255951 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:38.255850 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:18:40.535122 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:40.535091 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:18:49.017764 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:49.017727 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-699778cc4b-9cmcd"] Sep 19 04:18:49.165616 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:49.165563 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:18:49.263541 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:49.263506 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:18:49.408746 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:49.408709 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:18:56.631965 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:56.631928 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:18:56.734831 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:18:56.734789 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:19:14.037450 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.037386 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-699778cc4b-9cmcd" podUID="fd09ca20-dece-4514-8a03-1b4f13f22789" containerName="console" containerID="cri-o://06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53" gracePeriod=15 Sep 19 04:19:14.281989 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.281962 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:19:14.411980 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.411889 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-serving-cert\") pod \"fd09ca20-dece-4514-8a03-1b4f13f22789\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " Sep 19 04:19:14.411980 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.411931 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-oauth-serving-cert\") pod \"fd09ca20-dece-4514-8a03-1b4f13f22789\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " Sep 19 04:19:14.411980 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.411960 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-trusted-ca-bundle\") pod \"fd09ca20-dece-4514-8a03-1b4f13f22789\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " Sep 19 04:19:14.411980 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.411980 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tsft\" (UniqueName: \"kubernetes.io/projected/fd09ca20-dece-4514-8a03-1b4f13f22789-kube-api-access-7tsft\") pod \"fd09ca20-dece-4514-8a03-1b4f13f22789\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " Sep 19 04:19:14.412332 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.412222 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-oauth-config\") pod \"fd09ca20-dece-4514-8a03-1b4f13f22789\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " Sep 19 04:19:14.412332 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.412273 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-console-config\") pod \"fd09ca20-dece-4514-8a03-1b4f13f22789\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " Sep 19 04:19:14.412332 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.412325 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-service-ca\") pod \"fd09ca20-dece-4514-8a03-1b4f13f22789\" (UID: \"fd09ca20-dece-4514-8a03-1b4f13f22789\") " Sep 19 04:19:14.412465 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.412367 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fd09ca20-dece-4514-8a03-1b4f13f22789" (UID: "fd09ca20-dece-4514-8a03-1b4f13f22789"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:19:14.412608 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.412593 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-oauth-serving-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:19:14.412691 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.412649 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fd09ca20-dece-4514-8a03-1b4f13f22789" (UID: "fd09ca20-dece-4514-8a03-1b4f13f22789"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:19:14.412691 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.412643 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-console-config" (OuterVolumeSpecName: "console-config") pod "fd09ca20-dece-4514-8a03-1b4f13f22789" (UID: "fd09ca20-dece-4514-8a03-1b4f13f22789"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:19:14.412994 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.412972 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-service-ca" (OuterVolumeSpecName: "service-ca") pod "fd09ca20-dece-4514-8a03-1b4f13f22789" (UID: "fd09ca20-dece-4514-8a03-1b4f13f22789"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:19:14.414323 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.414293 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd09ca20-dece-4514-8a03-1b4f13f22789-kube-api-access-7tsft" (OuterVolumeSpecName: "kube-api-access-7tsft") pod "fd09ca20-dece-4514-8a03-1b4f13f22789" (UID: "fd09ca20-dece-4514-8a03-1b4f13f22789"). InnerVolumeSpecName "kube-api-access-7tsft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:19:14.414576 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.414557 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fd09ca20-dece-4514-8a03-1b4f13f22789" (UID: "fd09ca20-dece-4514-8a03-1b4f13f22789"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:19:14.414717 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.414696 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fd09ca20-dece-4514-8a03-1b4f13f22789" (UID: "fd09ca20-dece-4514-8a03-1b4f13f22789"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 04:19:14.514009 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.513971 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-oauth-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:19:14.514009 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.514006 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-console-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:19:14.514009 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.514016 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-service-ca\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:19:14.514275 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.514025 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd09ca20-dece-4514-8a03-1b4f13f22789-console-serving-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:19:14.514275 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.514033 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd09ca20-dece-4514-8a03-1b4f13f22789-trusted-ca-bundle\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:19:14.514275 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.514042 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tsft\" (UniqueName: \"kubernetes.io/projected/fd09ca20-dece-4514-8a03-1b4f13f22789-kube-api-access-7tsft\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:19:14.993366 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.993332 2577 generic.go:356] "Generic (PLEG): container finished" podID="fd09ca20-dece-4514-8a03-1b4f13f22789" containerID="06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53" exitCode=0 Sep 19 04:19:14.993541 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.993375 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-699778cc4b-9cmcd" Sep 19 04:19:14.993541 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.993418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-699778cc4b-9cmcd" event={"ID":"fd09ca20-dece-4514-8a03-1b4f13f22789","Type":"ContainerDied","Data":"06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53"} Sep 19 04:19:14.993541 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.993471 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-699778cc4b-9cmcd" event={"ID":"fd09ca20-dece-4514-8a03-1b4f13f22789","Type":"ContainerDied","Data":"6ad222bd4113a000baca8822e581dc3b004097da9f2aa539c5f36bf7953771f1"} Sep 19 04:19:14.993541 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:14.993493 2577 scope.go:117] "RemoveContainer" containerID="06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53" Sep 19 04:19:15.003375 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:15.003353 2577 scope.go:117] "RemoveContainer" containerID="06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53" Sep 19 04:19:15.003678 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:19:15.003657 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53\": container with ID starting with 06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53 not found: ID does not exist" containerID="06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53" Sep 19 04:19:15.003721 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:15.003686 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53"} err="failed to get container status \"06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53\": rpc error: code = NotFound desc = could not find container \"06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53\": container with ID starting with 06708d3d597ac78e77c8aae2bc43202fbccefe5be0a39d859c53e1cd1e12bd53 not found: ID does not exist" Sep 19 04:19:15.016035 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:15.016006 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-699778cc4b-9cmcd"] Sep 19 04:19:15.020129 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:15.020106 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-699778cc4b-9cmcd"] Sep 19 04:19:15.893682 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:15.893646 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd09ca20-dece-4514-8a03-1b4f13f22789" path="/var/lib/kubelet/pods/fd09ca20-dece-4514-8a03-1b4f13f22789/volumes" Sep 19 04:19:33.676642 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:33.676600 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:19:35.535794 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:35.535756 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:19:35.551687 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:35.551663 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:19:36.090139 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:36.090112 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 04:19:44.405714 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:44.405636 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:19:44.476118 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:44.476073 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:19:45.655833 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:45.655753 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:19:45.740163 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:45.740127 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:19:54.739045 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:54.739007 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:19:56.458699 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:56.458663 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:19:56.564247 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:56.564215 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:19:56.660132 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:19:56.660094 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:02.809338 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:02.809296 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:05.233643 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:05.233599 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:05.304996 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:05.304962 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:05.379301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:05.379269 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:16.853367 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:16.853335 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:16.918659 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:16.918618 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:18.685703 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:18.685669 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:35.752844 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:35.752815 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wkj2s_a151acea-4940-48b8-b206-571d1ea4fb4f/ovn-acl-logging/0.log" Sep 19 04:20:35.756686 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:35.756666 2577 kubelet.go:1648] "Image garbage collection succeeded" Sep 19 04:20:35.758574 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:35.758552 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wkj2s_a151acea-4940-48b8-b206-571d1ea4fb4f/ovn-acl-logging/0.log" Sep 19 04:20:36.021911 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:36.021880 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:37.216301 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:37.216265 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:20:58.768580 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:20:58.768545 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:00.178420 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:00.178384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:02.612722 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:02.612679 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:02.708369 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:02.708327 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:04.619808 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:04.619772 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:04.722710 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:04.722676 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:04.820813 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:04.820774 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:04.925917 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:04.925827 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:06.632373 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:06.632337 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:07.033455 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:07.033424 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:09.570953 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:09.570912 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:09.633636 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:09.633601 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:13.263020 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:13.262954 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:13.366179 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:13.366117 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:15.434755 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:15.434676 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:17.712909 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:17.712876 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:17.817139 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:17.817105 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:22.354029 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:22.353998 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:22.454040 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:22.454003 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:24.960853 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:24.960820 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:33.246530 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.246483 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:33.323183 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.323135 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7755db756b-trxfn"] Sep 19 04:21:33.323496 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.323483 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd09ca20-dece-4514-8a03-1b4f13f22789" containerName="console" Sep 19 04:21:33.323539 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.323502 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd09ca20-dece-4514-8a03-1b4f13f22789" containerName="console" Sep 19 04:21:33.323573 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.323555 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd09ca20-dece-4514-8a03-1b4f13f22789" containerName="console" Sep 19 04:21:33.326509 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.326491 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.330529 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:21:33.330500 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-cert\" is forbidden: User \"system:node:ip-10-0-132-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-132-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret" Sep 19 04:21:33.330907 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.330887 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 19 04:21:33.331379 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.331363 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 19 04:21:33.331466 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.331390 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wqlw4\"" Sep 19 04:21:33.331917 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.331836 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 19 04:21:33.331917 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.331860 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 19 04:21:33.351088 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.351065 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7755db756b-trxfn"] Sep 19 04:21:33.352832 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.352791 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:33.383592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.383570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a04db80-840b-4a5f-9d2a-e145a34815f4-metrics-cert\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.383698 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.383607 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqjmw\" (UniqueName: \"kubernetes.io/projected/6a04db80-840b-4a5f-9d2a-e145a34815f4-kube-api-access-kqjmw\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.383698 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.383625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6a04db80-840b-4a5f-9d2a-e145a34815f4-manager-config\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.383698 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.383654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a04db80-840b-4a5f-9d2a-e145a34815f4-cert\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.484401 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.484369 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjmw\" (UniqueName: \"kubernetes.io/projected/6a04db80-840b-4a5f-9d2a-e145a34815f4-kube-api-access-kqjmw\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.484401 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.484401 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6a04db80-840b-4a5f-9d2a-e145a34815f4-manager-config\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.484569 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.484450 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a04db80-840b-4a5f-9d2a-e145a34815f4-cert\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.484635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.484611 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a04db80-840b-4a5f-9d2a-e145a34815f4-metrics-cert\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.485101 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.485079 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6a04db80-840b-4a5f-9d2a-e145a34815f4-manager-config\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.486966 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.486945 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a04db80-840b-4a5f-9d2a-e145a34815f4-cert\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:33.494837 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:33.494815 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqjmw\" (UniqueName: \"kubernetes.io/projected/6a04db80-840b-4a5f-9d2a-e145a34815f4-kube-api-access-kqjmw\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:34.485691 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:21:34.485640 2577 secret.go:189] Couldn't get secret openshift-lws-operator/metrics-server-cert: failed to sync secret cache: timed out waiting for the condition Sep 19 04:21:34.486075 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:21:34.485732 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a04db80-840b-4a5f-9d2a-e145a34815f4-metrics-cert podName:6a04db80-840b-4a5f-9d2a-e145a34815f4 nodeName:}" failed. No retries permitted until 2026-09-19 04:21:34.98571523 +0000 UTC m=+361.306813355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-cert" (UniqueName: "kubernetes.io/secret/6a04db80-840b-4a5f-9d2a-e145a34815f4-metrics-cert") pod "lws-controller-manager-7755db756b-trxfn" (UID: "6a04db80-840b-4a5f-9d2a-e145a34815f4") : failed to sync secret cache: timed out waiting for the condition Sep 19 04:21:34.537869 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:34.537839 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 19 04:21:34.995643 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:34.995600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a04db80-840b-4a5f-9d2a-e145a34815f4-metrics-cert\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:34.998174 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:34.998122 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a04db80-840b-4a5f-9d2a-e145a34815f4-metrics-cert\") pod \"lws-controller-manager-7755db756b-trxfn\" (UID: \"6a04db80-840b-4a5f-9d2a-e145a34815f4\") " pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:35.135646 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:35.135604 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:35.412096 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:35.412071 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7755db756b-trxfn"] Sep 19 04:21:35.414071 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:21:35.414041 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a04db80_840b_4a5f_9d2a_e145a34815f4.slice/crio-a844fcf61157c765dce08760716af516729b25aef14a980465669999142ee171 WatchSource:0}: Error finding container a844fcf61157c765dce08760716af516729b25aef14a980465669999142ee171: Status 404 returned error can't find the container with id a844fcf61157c765dce08760716af516729b25aef14a980465669999142ee171 Sep 19 04:21:35.415992 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:35.415977 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 04:21:35.522096 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:35.522059 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" event={"ID":"6a04db80-840b-4a5f-9d2a-e145a34815f4","Type":"ContainerStarted","Data":"a844fcf61157c765dce08760716af516729b25aef14a980465669999142ee171"} Sep 19 04:21:36.246065 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:36.245986 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:36.341841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:36.341798 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:36.884520 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:36.884474 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:37.365767 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.365725 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:37.481729 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.481696 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:37.508259 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.508222 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9"] Sep 19 04:21:37.511791 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.511772 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.514351 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.514324 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 19 04:21:37.514479 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.514436 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-clzq7\"" Sep 19 04:21:37.515421 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.515393 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 19 04:21:37.515544 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.515497 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 19 04:21:37.523032 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.523012 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 19 04:21:37.534401 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.534373 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9"] Sep 19 04:21:37.618234 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.618212 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b825d08c-6b41-4bf0-9b48-345ab97d8eea-manifests\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.618362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.618241 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b825d08c-6b41-4bf0-9b48-345ab97d8eea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.618362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.618263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b825d08c-6b41-4bf0-9b48-345ab97d8eea-webhook-cert\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.618362 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.618285 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg6j4\" (UniqueName: \"kubernetes.io/projected/b825d08c-6b41-4bf0-9b48-345ab97d8eea-kube-api-access-fg6j4\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.618568 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.618383 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b825d08c-6b41-4bf0-9b48-345ab97d8eea-tmp\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.707621 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.707564 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:37.719741 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.719713 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b825d08c-6b41-4bf0-9b48-345ab97d8eea-tmp\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.719818 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.719779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b825d08c-6b41-4bf0-9b48-345ab97d8eea-manifests\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.719818 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.719796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b825d08c-6b41-4bf0-9b48-345ab97d8eea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.719818 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.719815 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b825d08c-6b41-4bf0-9b48-345ab97d8eea-webhook-cert\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.719961 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.719836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg6j4\" (UniqueName: \"kubernetes.io/projected/b825d08c-6b41-4bf0-9b48-345ab97d8eea-kube-api-access-fg6j4\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.720237 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.720214 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b825d08c-6b41-4bf0-9b48-345ab97d8eea-manifests\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.722423 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.722395 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b825d08c-6b41-4bf0-9b48-345ab97d8eea-webhook-cert\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.722600 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.722584 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b825d08c-6b41-4bf0-9b48-345ab97d8eea-tmp\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.722833 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.722817 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b825d08c-6b41-4bf0-9b48-345ab97d8eea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.740025 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.740001 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg6j4\" (UniqueName: \"kubernetes.io/projected/b825d08c-6b41-4bf0-9b48-345ab97d8eea-kube-api-access-fg6j4\") pod \"opendatahub-operator-controller-manager-78d8fdcb89-cwgg9\" (UID: \"b825d08c-6b41-4bf0-9b48-345ab97d8eea\") " pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.806511 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.806477 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:37.822565 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.822531 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:37.910725 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:37.910679 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:38.056840 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:38.056799 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:38.110972 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:38.110948 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9"] Sep 19 04:21:38.113658 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:21:38.113634 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb825d08c_6b41_4bf0_9b48_345ab97d8eea.slice/crio-963abe24d881226da77ffebdcccfd8c973570d788667a41561c117ddc1cbb617 WatchSource:0}: Error finding container 963abe24d881226da77ffebdcccfd8c973570d788667a41561c117ddc1cbb617: Status 404 returned error can't find the container with id 963abe24d881226da77ffebdcccfd8c973570d788667a41561c117ddc1cbb617 Sep 19 04:21:38.149743 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:38.149647 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:38.251232 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:38.251194 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:38.346087 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:38.346049 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:38.536682 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:38.536650 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" event={"ID":"6a04db80-840b-4a5f-9d2a-e145a34815f4","Type":"ContainerStarted","Data":"667b639c3f3fa75432510098fcbade5c2ab17a71c5271d4e8474be82d213dabf"} Sep 19 04:21:38.536889 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:38.536770 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:38.538135 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:38.538100 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" event={"ID":"b825d08c-6b41-4bf0-9b48-345ab97d8eea","Type":"ContainerStarted","Data":"963abe24d881226da77ffebdcccfd8c973570d788667a41561c117ddc1cbb617"} Sep 19 04:21:38.576098 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:38.576044 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" podStartSLOduration=3.4385603270000002 podStartE2EDuration="5.576023787s" totalImagesPullingTime="2.13746346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:21:38.574307154 +0000 UTC m=+364.895405297" watchObservedRunningTime="2026-09-19 04:21:38.576023787 +0000 UTC m=+364.897121933" Sep 19 04:21:41.554386 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:41.554351 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" event={"ID":"b825d08c-6b41-4bf0-9b48-345ab97d8eea","Type":"ContainerStarted","Data":"f9b601309762f3e50bc89fe2e3aaa4c2c218bd9f24f7d9937b607339d31c4e7e"} Sep 19 04:21:42.563172 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:42.563130 2577 generic.go:356] "Generic (PLEG): container finished" podID="b825d08c-6b41-4bf0-9b48-345ab97d8eea" containerID="f9b601309762f3e50bc89fe2e3aaa4c2c218bd9f24f7d9937b607339d31c4e7e" exitCode=0 Sep 19 04:21:42.563652 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:42.563214 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" event={"ID":"b825d08c-6b41-4bf0-9b48-345ab97d8eea","Type":"ContainerDied","Data":"f9b601309762f3e50bc89fe2e3aaa4c2c218bd9f24f7d9937b607339d31c4e7e"} Sep 19 04:21:43.569076 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:43.569043 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" event={"ID":"b825d08c-6b41-4bf0-9b48-345ab97d8eea","Type":"ContainerStarted","Data":"c86a833e4403a4e8d1cd3424a076517ec2470d71b43a9453119e7a7a99b27677"} Sep 19 04:21:43.569516 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:43.569131 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:43.645581 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:43.645520 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" podStartSLOduration=2.297614023 podStartE2EDuration="6.645502849s" totalImagesPullingTime="3.356591765s" totalInitContainerRuntime="991.297061ms" isStatefulPod=false podCreationTimestamp="2026-09-19 04:21:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:21:43.64236406 +0000 UTC m=+369.963462205" watchObservedRunningTime="2026-09-19 04:21:43.645502849 +0000 UTC m=+369.966600996" Sep 19 04:21:49.546428 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:49.546392 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7755db756b-trxfn" Sep 19 04:21:54.574916 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:54.574882 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78d8fdcb89-cwgg9" Sep 19 04:21:57.032831 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.032790 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:57.033263 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.032946 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64cfdb4dc9-scgkc"] Sep 19 04:21:57.153444 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.153406 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cfdb4dc9-scgkc"] Sep 19 04:21:57.153604 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.153546 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.164030 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.164004 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 19 04:21:57.164148 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.164013 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 19 04:21:57.164744 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.164728 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 19 04:21:57.164853 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.164835 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9h7ml\"" Sep 19 04:21:57.164895 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.164875 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 19 04:21:57.164945 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.164840 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 19 04:21:57.165074 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.165057 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 19 04:21:57.169814 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.169798 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 19 04:21:57.170102 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.170088 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 19 04:21:57.192735 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.192707 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mmjk\" (UniqueName: \"kubernetes.io/projected/80b4d933-4265-4f0f-8086-965ffcb6964b-kube-api-access-4mmjk\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.192882 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.192740 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b4d933-4265-4f0f-8086-965ffcb6964b-console-serving-cert\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.192882 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.192770 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-oauth-serving-cert\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.192882 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.192846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80b4d933-4265-4f0f-8086-965ffcb6964b-console-oauth-config\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.193032 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.192880 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-console-config\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.193032 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.192939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-service-ca\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.193032 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.193019 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-trusted-ca-bundle\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.222292 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.222256 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:57.293514 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.293439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80b4d933-4265-4f0f-8086-965ffcb6964b-console-oauth-config\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.293514 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.293472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-console-config\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.293702 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.293647 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-service-ca\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.293756 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.293714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-trusted-ca-bundle\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.293805 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.293790 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mmjk\" (UniqueName: \"kubernetes.io/projected/80b4d933-4265-4f0f-8086-965ffcb6964b-kube-api-access-4mmjk\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.293841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.293825 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b4d933-4265-4f0f-8086-965ffcb6964b-console-serving-cert\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.293978 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.293861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-oauth-serving-cert\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.294387 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.294364 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-service-ca\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.294470 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.294451 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-console-config\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.294644 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.294545 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-oauth-serving-cert\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.294644 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.294622 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80b4d933-4265-4f0f-8086-965ffcb6964b-trusted-ca-bundle\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.296071 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.296048 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80b4d933-4265-4f0f-8086-965ffcb6964b-console-oauth-config\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.296947 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.296925 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b4d933-4265-4f0f-8086-965ffcb6964b-console-serving-cert\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.336447 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.336417 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mmjk\" (UniqueName: \"kubernetes.io/projected/80b4d933-4265-4f0f-8086-965ffcb6964b-kube-api-access-4mmjk\") pod \"console-64cfdb4dc9-scgkc\" (UID: \"80b4d933-4265-4f0f-8086-965ffcb6964b\") " pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.369398 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.369361 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:57.462802 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.462770 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:21:57.654200 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.654098 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:57.719540 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.719506 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss"] Sep 19 04:21:57.742593 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.742563 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:57.745575 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.745552 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 19 04:21:57.746181 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.746165 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-q2jnw\"" Sep 19 04:21:57.752724 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.752701 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 19 04:21:57.752835 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.752782 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss"] Sep 19 04:21:57.798555 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:21:57.798526 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80b4d933_4265_4f0f_8086_965ffcb6964b.slice/crio-3e1c5e3cd17a02adb9139d60314066ec017bd2051edc32aebf0904c79e445272 WatchSource:0}: Error finding container 3e1c5e3cd17a02adb9139d60314066ec017bd2051edc32aebf0904c79e445272: Status 404 returned error can't find the container with id 3e1c5e3cd17a02adb9139d60314066ec017bd2051edc32aebf0904c79e445272 Sep 19 04:21:57.798664 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.798624 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5-tls-certs\") pod \"kube-auth-proxy-77df8579b6-fd2ss\" (UID: \"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5\") " pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:57.798713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.798683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldksh\" (UniqueName: \"kubernetes.io/projected/b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5-kube-api-access-ldksh\") pod \"kube-auth-proxy-77df8579b6-fd2ss\" (UID: \"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5\") " pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:57.798754 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.798744 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5-tmp\") pod \"kube-auth-proxy-77df8579b6-fd2ss\" (UID: \"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5\") " pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:57.804987 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.804962 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cfdb4dc9-scgkc"] Sep 19 04:21:57.899245 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.899214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5-tls-certs\") pod \"kube-auth-proxy-77df8579b6-fd2ss\" (UID: \"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5\") " pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:57.899371 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.899319 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldksh\" (UniqueName: \"kubernetes.io/projected/b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5-kube-api-access-ldksh\") pod \"kube-auth-proxy-77df8579b6-fd2ss\" (UID: \"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5\") " pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:57.899436 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.899403 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5-tmp\") pod \"kube-auth-proxy-77df8579b6-fd2ss\" (UID: \"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5\") " pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:57.901673 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.901651 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5-tmp\") pod \"kube-auth-proxy-77df8579b6-fd2ss\" (UID: \"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5\") " pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:57.901881 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.901865 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5-tls-certs\") pod \"kube-auth-proxy-77df8579b6-fd2ss\" (UID: \"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5\") " pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:57.917049 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:57.916965 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldksh\" (UniqueName: \"kubernetes.io/projected/b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5-kube-api-access-ldksh\") pod \"kube-auth-proxy-77df8579b6-fd2ss\" (UID: \"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5\") " pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:58.000898 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:58.000862 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:58.053132 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:58.053101 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" Sep 19 04:21:58.349313 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:58.349267 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss"] Sep 19 04:21:58.353205 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:21:58.353172 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6ab50b8_7a19_4c1a_8c3b_f3f89e5bb5e5.slice/crio-72a7edfbfc853a01cd4ccdc70b72626783507332ba31d9b54d0b46c023bc69b7 WatchSource:0}: Error finding container 72a7edfbfc853a01cd4ccdc70b72626783507332ba31d9b54d0b46c023bc69b7: Status 404 returned error can't find the container with id 72a7edfbfc853a01cd4ccdc70b72626783507332ba31d9b54d0b46c023bc69b7 Sep 19 04:21:58.631993 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:58.631896 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cfdb4dc9-scgkc" event={"ID":"80b4d933-4265-4f0f-8086-965ffcb6964b","Type":"ContainerStarted","Data":"23cd9e5ffce51104b9f7a838c511eed7a440b9aa30dcf474369822264f6c1b43"} Sep 19 04:21:58.631993 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:58.631983 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cfdb4dc9-scgkc" event={"ID":"80b4d933-4265-4f0f-8086-965ffcb6964b","Type":"ContainerStarted","Data":"3e1c5e3cd17a02adb9139d60314066ec017bd2051edc32aebf0904c79e445272"} Sep 19 04:21:58.633102 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:58.633080 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" event={"ID":"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5","Type":"ContainerStarted","Data":"72a7edfbfc853a01cd4ccdc70b72626783507332ba31d9b54d0b46c023bc69b7"} Sep 19 04:21:59.663497 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:59.663460 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:59.849947 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:59.849906 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:21:59.961488 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:21:59.961391 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:01.650009 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:01.649978 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" event={"ID":"b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5","Type":"ContainerStarted","Data":"8d50dbd3363bd77cc10ac3bf8ba0203e37fe74180722b38408cdfcc98f18c542"} Sep 19 04:22:01.691661 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:01.691598 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64cfdb4dc9-scgkc" podStartSLOduration=5.691581034 podStartE2EDuration="5.691581034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:21:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:21:58.67243339 +0000 UTC m=+384.993531537" watchObservedRunningTime="2026-09-19 04:22:01.691581034 +0000 UTC m=+388.012679177" Sep 19 04:22:01.813584 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:01.813536 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:04.490374 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:04.490329 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:04.589546 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:04.589503 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:05.282405 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:05.282355 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:05.537064 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:05.536962 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:06.017777 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:06.017694 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:07.463126 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:07.463090 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:22:07.463537 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:07.463148 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:22:07.467808 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:07.467784 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:22:07.489183 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:07.489091 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77df8579b6-fd2ss" podStartSLOduration=7.709570082 podStartE2EDuration="10.489070459s" totalImagesPullingTime="2.779500377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:21:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:22:01.692444848 +0000 UTC m=+388.013542993" watchObservedRunningTime="2026-09-19 04:22:07.489070459 +0000 UTC m=+393.810168606" Sep 19 04:22:07.682898 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:07.682866 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64cfdb4dc9-scgkc" Sep 19 04:22:07.740596 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:07.740513 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:07.876601 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:07.876565 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:07.979713 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:07.979677 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:08.080221 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:08.080181 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:08.345560 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:08.345477 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:08.442395 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:08.442342 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:08.778422 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:08.778385 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:19.695754 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:19.695708 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:19.851592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:19.851557 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:20.079380 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:20.079351 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:20.218563 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:20.218520 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:20.358347 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:20.358253 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:21.340407 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:21.340368 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:21.465692 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:21.465652 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:21.542958 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:21.542920 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:36.637510 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:36.637467 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:22:36.730421 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:22:36.730384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:08.854423 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:08.854371 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:09.005435 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:09.005387 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:28.721731 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:28.721686 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:31.316916 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:31.316882 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:31.759411 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:31.759314 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:32.196175 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:32.196122 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:33.415555 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:33.415519 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:34.903046 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:34.903001 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:35.013194 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:35.013143 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:36.308589 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:36.308552 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:37.345233 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:37.345186 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:38.110927 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:38.110888 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:38.514906 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:38.514810 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:39.600926 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:39.600884 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:39.750981 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:39.750942 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:43.331570 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.331538 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:43.445467 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.445433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:43.837324 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.837285 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:43.857225 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.857194 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb"] Sep 19 04:23:43.860986 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.860947 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:43.864793 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.864767 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7xdvw\"" Sep 19 04:23:43.865256 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.865236 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 19 04:23:43.865405 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.865383 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 19 04:23:43.865672 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.865653 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 19 04:23:43.865790 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.865776 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 19 04:23:43.871373 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.871349 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb"] Sep 19 04:23:43.958952 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.958918 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:43.967364 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.967341 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ef877b8d-5f04-4c5e-8542-5104dd3e83ab-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-tqqpb\" (UID: \"ef877b8d-5f04-4c5e-8542-5104dd3e83ab\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:43.967471 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.967375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef877b8d-5f04-4c5e-8542-5104dd3e83ab-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-tqqpb\" (UID: \"ef877b8d-5f04-4c5e-8542-5104dd3e83ab\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:43.967471 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:43.967408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdlm9\" (UniqueName: \"kubernetes.io/projected/ef877b8d-5f04-4c5e-8542-5104dd3e83ab-kube-api-access-wdlm9\") pod \"kuadrant-console-plugin-fbdd4f445-tqqpb\" (UID: \"ef877b8d-5f04-4c5e-8542-5104dd3e83ab\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:44.068086 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:44.068053 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ef877b8d-5f04-4c5e-8542-5104dd3e83ab-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-tqqpb\" (UID: \"ef877b8d-5f04-4c5e-8542-5104dd3e83ab\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:44.068256 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:44.068093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef877b8d-5f04-4c5e-8542-5104dd3e83ab-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-tqqpb\" (UID: \"ef877b8d-5f04-4c5e-8542-5104dd3e83ab\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:44.068256 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:44.068131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdlm9\" (UniqueName: \"kubernetes.io/projected/ef877b8d-5f04-4c5e-8542-5104dd3e83ab-kube-api-access-wdlm9\") pod \"kuadrant-console-plugin-fbdd4f445-tqqpb\" (UID: \"ef877b8d-5f04-4c5e-8542-5104dd3e83ab\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:44.068687 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:44.068665 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ef877b8d-5f04-4c5e-8542-5104dd3e83ab-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-tqqpb\" (UID: \"ef877b8d-5f04-4c5e-8542-5104dd3e83ab\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:44.070702 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:44.070685 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef877b8d-5f04-4c5e-8542-5104dd3e83ab-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-tqqpb\" (UID: \"ef877b8d-5f04-4c5e-8542-5104dd3e83ab\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:44.077867 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:44.077841 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdlm9\" (UniqueName: \"kubernetes.io/projected/ef877b8d-5f04-4c5e-8542-5104dd3e83ab-kube-api-access-wdlm9\") pod \"kuadrant-console-plugin-fbdd4f445-tqqpb\" (UID: \"ef877b8d-5f04-4c5e-8542-5104dd3e83ab\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:44.185274 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:44.185194 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" Sep 19 04:23:44.472051 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:44.472027 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb"] Sep 19 04:23:44.474639 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:23:44.474602 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef877b8d_5f04_4c5e_8542_5104dd3e83ab.slice/crio-43403098b5db46b7ea50dd90d7b3a3334f0e53bfa5db9e534293e2859f78117e WatchSource:0}: Error finding container 43403098b5db46b7ea50dd90d7b3a3334f0e53bfa5db9e534293e2859f78117e: Status 404 returned error can't find the container with id 43403098b5db46b7ea50dd90d7b3a3334f0e53bfa5db9e534293e2859f78117e Sep 19 04:23:45.103927 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:45.103885 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" event={"ID":"ef877b8d-5f04-4c5e-8542-5104dd3e83ab","Type":"ContainerStarted","Data":"43403098b5db46b7ea50dd90d7b3a3334f0e53bfa5db9e534293e2859f78117e"} Sep 19 04:23:45.308353 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:45.308122 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:45.407824 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:45.407631 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:48.291244 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:48.291197 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:48.392195 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:48.392128 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:48.721446 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:48.721350 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:48.821683 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:48.821643 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:49.094661 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:49.094636 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:49.124044 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:49.124014 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" event={"ID":"ef877b8d-5f04-4c5e-8542-5104dd3e83ab","Type":"ContainerStarted","Data":"af075faadcd750585be35887b73028cb8dff73c4f610391842f22e6ffc15a7f6"} Sep 19 04:23:49.145461 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:49.145411 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tqqpb" podStartSLOduration=1.586055279 podStartE2EDuration="6.145396475s" totalImagesPullingTime="4.559341196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:23:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:23:49.144069971 +0000 UTC m=+495.465168124" watchObservedRunningTime="2026-09-19 04:23:49.145396475 +0000 UTC m=+495.466494620" Sep 19 04:23:49.196118 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:49.196084 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:55.442805 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:55.442765 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:55.572304 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:55.572275 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:23:56.670555 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.670521 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj"] Sep 19 04:23:56.674051 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.674034 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:23:56.677658 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.677635 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f7vns\"" Sep 19 04:23:56.682304 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.682282 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b791d0e1-1512-423b-bfa8-d9b94bc9862e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vtrkj\" (UID: \"b791d0e1-1512-423b-bfa8-d9b94bc9862e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:23:56.682421 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.682346 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxzkc\" (UniqueName: \"kubernetes.io/projected/b791d0e1-1512-423b-bfa8-d9b94bc9862e-kube-api-access-vxzkc\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vtrkj\" (UID: \"b791d0e1-1512-423b-bfa8-d9b94bc9862e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:23:56.701190 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.701142 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj"] Sep 19 04:23:56.783448 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.783410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxzkc\" (UniqueName: \"kubernetes.io/projected/b791d0e1-1512-423b-bfa8-d9b94bc9862e-kube-api-access-vxzkc\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vtrkj\" (UID: \"b791d0e1-1512-423b-bfa8-d9b94bc9862e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:23:56.783617 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.783493 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b791d0e1-1512-423b-bfa8-d9b94bc9862e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vtrkj\" (UID: \"b791d0e1-1512-423b-bfa8-d9b94bc9862e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:23:56.783870 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.783852 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b791d0e1-1512-423b-bfa8-d9b94bc9862e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vtrkj\" (UID: \"b791d0e1-1512-423b-bfa8-d9b94bc9862e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:23:56.806277 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.806243 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxzkc\" (UniqueName: \"kubernetes.io/projected/b791d0e1-1512-423b-bfa8-d9b94bc9862e-kube-api-access-vxzkc\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vtrkj\" (UID: \"b791d0e1-1512-423b-bfa8-d9b94bc9862e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:23:56.984449 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:56.984359 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:23:57.280332 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:57.280195 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj"] Sep 19 04:23:57.283437 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:23:57.283405 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb791d0e1_1512_423b_bfa8_d9b94bc9862e.slice/crio-eaf88ff9936a594a30f14ea37bdfb8f40aa829eb6d50ed89735687b1c7362df0 WatchSource:0}: Error finding container eaf88ff9936a594a30f14ea37bdfb8f40aa829eb6d50ed89735687b1c7362df0: Status 404 returned error can't find the container with id eaf88ff9936a594a30f14ea37bdfb8f40aa829eb6d50ed89735687b1c7362df0 Sep 19 04:23:58.167742 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:23:58.167693 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" event={"ID":"b791d0e1-1512-423b-bfa8-d9b94bc9862e","Type":"ContainerStarted","Data":"eaf88ff9936a594a30f14ea37bdfb8f40aa829eb6d50ed89735687b1c7362df0"} Sep 19 04:24:00.162784 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:00.162746 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:00.268444 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:00.268391 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:01.183693 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:01.183648 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" event={"ID":"b791d0e1-1512-423b-bfa8-d9b94bc9862e","Type":"ContainerStarted","Data":"78c04d965cf438dc6e7e88e04f97d7801fe88f5e4626f7a7a5483043b8cf1e32"} Sep 19 04:24:01.184054 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:01.183715 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:24:01.241710 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:01.241656 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" podStartSLOduration=1.532302705 podStartE2EDuration="5.241639571s" totalImagesPullingTime="3.709336866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:24:01.240313453 +0000 UTC m=+507.561411598" watchObservedRunningTime="2026-09-19 04:24:01.241639571 +0000 UTC m=+507.562737716" Sep 19 04:24:02.860316 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:02.860267 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:02.976269 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:02.976217 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:08.144419 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:08.144374 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:08.242874 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:08.242835 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:08.546146 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:08.546104 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:08.647555 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:08.647525 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:08.940883 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:08.940783 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:09.042662 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:09.042624 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:09.568426 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:09.568388 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:09.698171 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:09.698107 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:12.190356 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:12.190322 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vtrkj" Sep 19 04:24:15.191605 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:15.191512 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:15.317425 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:15.317369 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:20.028490 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:20.028448 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:20.128587 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:20.128546 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:20.221591 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:20.221548 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:28.971312 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:28.971267 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:29.222853 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:29.222747 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:31.762146 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:31.762113 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:31.880556 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:31.880507 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:31.960070 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:31.960034 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:32.712967 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:32.712929 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:32.843121 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:32.843081 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:33.136618 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:33.136587 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-msfsx"] Sep 19 04:24:33.328291 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:33.328256 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-msfsx"] Sep 19 04:24:33.328472 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:33.328361 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-msfsx" Sep 19 04:24:33.331850 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:33.331830 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hcjkn\"" Sep 19 04:24:33.500339 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:33.500255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htx69\" (UniqueName: \"kubernetes.io/projected/df963a3b-bb66-4180-bc77-22ff87cbff04-kube-api-access-htx69\") pod \"authorino-59c974bd48-msfsx\" (UID: \"df963a3b-bb66-4180-bc77-22ff87cbff04\") " pod="kuadrant-system/authorino-59c974bd48-msfsx" Sep 19 04:24:33.601026 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:33.600988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htx69\" (UniqueName: \"kubernetes.io/projected/df963a3b-bb66-4180-bc77-22ff87cbff04-kube-api-access-htx69\") pod \"authorino-59c974bd48-msfsx\" (UID: \"df963a3b-bb66-4180-bc77-22ff87cbff04\") " pod="kuadrant-system/authorino-59c974bd48-msfsx" Sep 19 04:24:33.611488 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:33.611462 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htx69\" (UniqueName: \"kubernetes.io/projected/df963a3b-bb66-4180-bc77-22ff87cbff04-kube-api-access-htx69\") pod \"authorino-59c974bd48-msfsx\" (UID: \"df963a3b-bb66-4180-bc77-22ff87cbff04\") " pod="kuadrant-system/authorino-59c974bd48-msfsx" Sep 19 04:24:33.637383 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:33.637357 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-msfsx" Sep 19 04:24:33.926169 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:33.926116 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-msfsx"] Sep 19 04:24:33.930540 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:24:33.930510 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf963a3b_bb66_4180_bc77_22ff87cbff04.slice/crio-d20b3076e97b1bad7eb9b0e9a26c802281dedc0e7e7b1a7eec1b6829c86c6593 WatchSource:0}: Error finding container d20b3076e97b1bad7eb9b0e9a26c802281dedc0e7e7b1a7eec1b6829c86c6593: Status 404 returned error can't find the container with id d20b3076e97b1bad7eb9b0e9a26c802281dedc0e7e7b1a7eec1b6829c86c6593 Sep 19 04:24:34.331773 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:34.331735 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-msfsx" event={"ID":"df963a3b-bb66-4180-bc77-22ff87cbff04","Type":"ContainerStarted","Data":"d20b3076e97b1bad7eb9b0e9a26c802281dedc0e7e7b1a7eec1b6829c86c6593"} Sep 19 04:24:36.342453 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:36.342413 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-msfsx" event={"ID":"df963a3b-bb66-4180-bc77-22ff87cbff04","Type":"ContainerStarted","Data":"04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4"} Sep 19 04:24:47.793521 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:47.793484 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:47.895511 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:47.895479 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:24:48.334419 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:24:48.334379 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:00.181663 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:00.181620 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:00.615057 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:00.615023 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:02.871842 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:02.871788 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-msfsx" podStartSLOduration=28.311462222 podStartE2EDuration="29.871772109s" totalImagesPullingTime="1.560309887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:24:36.365869692 +0000 UTC m=+542.686967837" watchObservedRunningTime="2026-09-19 04:25:02.871772109 +0000 UTC m=+569.192870255" Sep 19 04:25:02.873011 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:02.872993 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-msfsx"] Sep 19 04:25:02.873197 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:02.873177 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-msfsx" podUID="df963a3b-bb66-4180-bc77-22ff87cbff04" containerName="authorino" containerID="cri-o://04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4" gracePeriod=30 Sep 19 04:25:03.110380 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.110351 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-msfsx" Sep 19 04:25:03.160098 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.160023 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htx69\" (UniqueName: \"kubernetes.io/projected/df963a3b-bb66-4180-bc77-22ff87cbff04-kube-api-access-htx69\") pod \"df963a3b-bb66-4180-bc77-22ff87cbff04\" (UID: \"df963a3b-bb66-4180-bc77-22ff87cbff04\") " Sep 19 04:25:03.162442 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.162357 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df963a3b-bb66-4180-bc77-22ff87cbff04-kube-api-access-htx69" (OuterVolumeSpecName: "kube-api-access-htx69") pod "df963a3b-bb66-4180-bc77-22ff87cbff04" (UID: "df963a3b-bb66-4180-bc77-22ff87cbff04"). InnerVolumeSpecName "kube-api-access-htx69". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:25:03.261263 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.261227 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htx69\" (UniqueName: \"kubernetes.io/projected/df963a3b-bb66-4180-bc77-22ff87cbff04-kube-api-access-htx69\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:25:03.463218 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.463108 2577 generic.go:356] "Generic (PLEG): container finished" podID="df963a3b-bb66-4180-bc77-22ff87cbff04" containerID="04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4" exitCode=0 Sep 19 04:25:03.463218 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.463176 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-msfsx" Sep 19 04:25:03.463402 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.463245 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-msfsx" event={"ID":"df963a3b-bb66-4180-bc77-22ff87cbff04","Type":"ContainerDied","Data":"04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4"} Sep 19 04:25:03.463402 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.463295 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-msfsx" event={"ID":"df963a3b-bb66-4180-bc77-22ff87cbff04","Type":"ContainerDied","Data":"d20b3076e97b1bad7eb9b0e9a26c802281dedc0e7e7b1a7eec1b6829c86c6593"} Sep 19 04:25:03.463402 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.463318 2577 scope.go:117] "RemoveContainer" containerID="04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4" Sep 19 04:25:03.472340 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.472321 2577 scope.go:117] "RemoveContainer" containerID="04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4" Sep 19 04:25:03.472599 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:25:03.472580 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4\": container with ID starting with 04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4 not found: ID does not exist" containerID="04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4" Sep 19 04:25:03.472638 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.472610 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4"} err="failed to get container status \"04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4\": rpc error: code = NotFound desc = could not find container \"04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4\": container with ID starting with 04ef30a7e2f748ecf88fdf2241bc97d95fb97351f0473b6cbbd3c109e12a3db4 not found: ID does not exist" Sep 19 04:25:03.490894 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.490873 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-msfsx"] Sep 19 04:25:03.494148 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.494129 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-msfsx"] Sep 19 04:25:03.893855 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:03.893821 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df963a3b-bb66-4180-bc77-22ff87cbff04" path="/var/lib/kubelet/pods/df963a3b-bb66-4180-bc77-22ff87cbff04/volumes" Sep 19 04:25:07.858966 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:07.858925 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:08.184400 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:08.184309 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:11.192313 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:11.192269 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:11.304641 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:11.304601 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:35.782608 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:35.782581 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wkj2s_a151acea-4940-48b8-b206-571d1ea4fb4f/ovn-acl-logging/0.log" Sep 19 04:25:35.785364 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:35.785341 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wkj2s_a151acea-4940-48b8-b206-571d1ea4fb4f/ovn-acl-logging/0.log" Sep 19 04:25:44.391620 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:44.391574 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:45.455180 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:45.455078 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:45.972909 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:45.972873 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:46.154572 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:46.154536 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:46.290118 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:46.290061 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:46.404474 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:46.404424 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:46.828766 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:46.828730 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:46.958693 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:46.958643 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:47.124995 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.124920 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw"] Sep 19 04:25:47.125254 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.125242 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df963a3b-bb66-4180-bc77-22ff87cbff04" containerName="authorino" Sep 19 04:25:47.125323 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.125256 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="df963a3b-bb66-4180-bc77-22ff87cbff04" containerName="authorino" Sep 19 04:25:47.125360 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.125332 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="df963a3b-bb66-4180-bc77-22ff87cbff04" containerName="authorino" Sep 19 04:25:47.127425 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.127410 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:25:47.130848 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.130831 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 19 04:25:47.130956 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.130890 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hw6x4\"" Sep 19 04:25:47.215483 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.215445 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw"] Sep 19 04:25:47.248207 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.248172 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckq5r\" (UniqueName: \"kubernetes.io/projected/99ce8be0-22ad-4343-99b1-187ffc142a90-kube-api-access-ckq5r\") pod \"payload-pre-processing-59fbc47bf8-9v8gw\" (UID: \"99ce8be0-22ad-4343-99b1-187ffc142a90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:25:47.248389 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.248285 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ce8be0-22ad-4343-99b1-187ffc142a90-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9v8gw\" (UID: \"99ce8be0-22ad-4343-99b1-187ffc142a90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:25:47.349038 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.349008 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckq5r\" (UniqueName: \"kubernetes.io/projected/99ce8be0-22ad-4343-99b1-187ffc142a90-kube-api-access-ckq5r\") pod \"payload-pre-processing-59fbc47bf8-9v8gw\" (UID: \"99ce8be0-22ad-4343-99b1-187ffc142a90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:25:47.349230 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.349081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ce8be0-22ad-4343-99b1-187ffc142a90-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9v8gw\" (UID: \"99ce8be0-22ad-4343-99b1-187ffc142a90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:25:47.349729 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.349710 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ce8be0-22ad-4343-99b1-187ffc142a90-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9v8gw\" (UID: \"99ce8be0-22ad-4343-99b1-187ffc142a90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:25:47.385694 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.385622 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckq5r\" (UniqueName: \"kubernetes.io/projected/99ce8be0-22ad-4343-99b1-187ffc142a90-kube-api-access-ckq5r\") pod \"payload-pre-processing-59fbc47bf8-9v8gw\" (UID: \"99ce8be0-22ad-4343-99b1-187ffc142a90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:25:47.437604 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.437567 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:25:47.733841 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:47.733754 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw"] Sep 19 04:25:47.736679 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:25:47.736644 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ce8be0_22ad_4343_99b1_187ffc142a90.slice/crio-aa6fbda6a350b1b67b2b6709ef501406d20ac720f83540a9b0c566c52c0fd2ee WatchSource:0}: Error finding container aa6fbda6a350b1b67b2b6709ef501406d20ac720f83540a9b0c566c52c0fd2ee: Status 404 returned error can't find the container with id aa6fbda6a350b1b67b2b6709ef501406d20ac720f83540a9b0c566c52c0fd2ee Sep 19 04:25:48.677200 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:48.677146 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" event={"ID":"99ce8be0-22ad-4343-99b1-187ffc142a90","Type":"ContainerStarted","Data":"aa6fbda6a350b1b67b2b6709ef501406d20ac720f83540a9b0c566c52c0fd2ee"} Sep 19 04:25:50.688454 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:50.688418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" event={"ID":"99ce8be0-22ad-4343-99b1-187ffc142a90","Type":"ContainerStarted","Data":"bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5"} Sep 19 04:25:50.688872 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:50.688586 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:25:50.708516 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:50.708461 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" podStartSLOduration=1.33236045 podStartE2EDuration="3.708441801s" totalImagesPullingTime="2.376081351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:25:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:25:50.706959059 +0000 UTC m=+617.028057206" watchObservedRunningTime="2026-09-19 04:25:50.708441801 +0000 UTC m=+617.029539948" Sep 19 04:25:55.140581 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:55.140546 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:55.247753 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:55.247709 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:56.110130 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:56.110092 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:25:56.211521 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:25:56.211479 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:26:01.694348 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:01.694310 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:26:09.803777 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:09.803738 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:26:23.817182 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.817123 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr"] Sep 19 04:26:23.825326 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.825301 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.828930 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.828903 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-h5pwl\"" Sep 19 04:26:23.829071 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.828938 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 19 04:26:23.829071 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.828942 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 19 04:26:23.829071 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.829006 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 19 04:26:23.834701 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.834680 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr"] Sep 19 04:26:23.843610 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.843584 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:26:23.862868 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.862837 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.862868 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.862865 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.863032 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.862896 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.863032 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.862958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30446db5-4131-4338-a2ec-1dee107d99bc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.863032 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.863013 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.863133 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.863037 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr7dj\" (UniqueName: \"kubernetes.io/projected/30446db5-4131-4338-a2ec-1dee107d99bc-kube-api-access-kr7dj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.964001 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.963964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.964001 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.964005 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.964233 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.964134 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.964233 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.964192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30446db5-4131-4338-a2ec-1dee107d99bc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.964233 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.964215 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.964368 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.964242 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr7dj\" (UniqueName: \"kubernetes.io/projected/30446db5-4131-4338-a2ec-1dee107d99bc-kube-api-access-kr7dj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.964417 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.964371 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.964565 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.964544 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.964633 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.964578 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.966521 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.966502 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30446db5-4131-4338-a2ec-1dee107d99bc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.966762 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.966744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30446db5-4131-4338-a2ec-1dee107d99bc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:23.980488 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:23.980463 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr7dj\" (UniqueName: \"kubernetes.io/projected/30446db5-4131-4338-a2ec-1dee107d99bc-kube-api-access-kr7dj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5pxwr\" (UID: \"30446db5-4131-4338-a2ec-1dee107d99bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:24.136461 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:24.136374 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:24.411901 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:24.411825 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr"] Sep 19 04:26:24.415021 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:26:24.414993 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30446db5_4131_4338_a2ec_1dee107d99bc.slice/crio-746713fa5d287dfbdec36012809a2aff91a9e1e3c8f0acab053aeaafd721c33d WatchSource:0}: Error finding container 746713fa5d287dfbdec36012809a2aff91a9e1e3c8f0acab053aeaafd721c33d: Status 404 returned error can't find the container with id 746713fa5d287dfbdec36012809a2aff91a9e1e3c8f0acab053aeaafd721c33d Sep 19 04:26:24.838992 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:24.838948 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" event={"ID":"30446db5-4131-4338-a2ec-1dee107d99bc","Type":"ContainerStarted","Data":"746713fa5d287dfbdec36012809a2aff91a9e1e3c8f0acab053aeaafd721c33d"} Sep 19 04:26:25.844446 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:25.844410 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" event={"ID":"30446db5-4131-4338-a2ec-1dee107d99bc","Type":"ContainerStarted","Data":"67eede937fc451512b346c19b67f13a98f043469daf958ff753efe63249898c8"} Sep 19 04:26:25.867811 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:25.867762 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" podStartSLOduration=1.8621365079999999 podStartE2EDuration="2.867745767s" totalImagesPullingTime="1.005609259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:26:25.865851347 +0000 UTC m=+652.186949494" watchObservedRunningTime="2026-09-19 04:26:25.867745767 +0000 UTC m=+652.188843912" Sep 19 04:26:34.137208 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:34.137146 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:34.137640 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:34.137245 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:34.149788 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:34.149766 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:34.896082 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:34.896054 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5pxwr" Sep 19 04:26:36.409397 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.409363 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr"] Sep 19 04:26:36.412822 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.412802 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.415519 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.415498 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 19 04:26:36.429049 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.429029 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr"] Sep 19 04:26:36.441660 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.441636 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:26:36.475332 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.475307 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.475483 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.475343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc54592-cb93-45bc-83d5-a9619cce5a0f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.475483 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.475361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shxnh\" (UniqueName: \"kubernetes.io/projected/4fc54592-cb93-45bc-83d5-a9619cce5a0f-kube-api-access-shxnh\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.475483 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.475379 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.475646 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.475491 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.475646 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.475538 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.576109 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.576067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.576109 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.576115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.576408 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.576204 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.576408 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.576228 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc54592-cb93-45bc-83d5-a9619cce5a0f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.576408 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.576249 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shxnh\" (UniqueName: \"kubernetes.io/projected/4fc54592-cb93-45bc-83d5-a9619cce5a0f-kube-api-access-shxnh\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.576408 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.576277 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.576667 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.576641 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.576792 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.576680 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.576851 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.576819 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.578653 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.578629 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc54592-cb93-45bc-83d5-a9619cce5a0f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.578815 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.578797 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc54592-cb93-45bc-83d5-a9619cce5a0f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.588425 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.588405 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shxnh\" (UniqueName: \"kubernetes.io/projected/4fc54592-cb93-45bc-83d5-a9619cce5a0f-kube-api-access-shxnh\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr\" (UID: \"4fc54592-cb93-45bc-83d5-a9619cce5a0f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.722737 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.722649 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:36.997140 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:36.997117 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr"] Sep 19 04:26:37.001809 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:37.001791 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 04:26:37.899096 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:37.899051 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" event={"ID":"4fc54592-cb93-45bc-83d5-a9619cce5a0f","Type":"ContainerStarted","Data":"231b8fc7d8c74ccad7fc05927143e0700eb6039d6bda8f76b03190a77c7db657"} Sep 19 04:26:37.899589 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:37.899110 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" event={"ID":"4fc54592-cb93-45bc-83d5-a9619cce5a0f","Type":"ContainerStarted","Data":"2cfd0d12129a6ebcd4a3db9bb233dff6428074a5277cfdbf545fb09103331016"} Sep 19 04:26:37.924348 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:37.924292 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" podStartSLOduration=1.749086627 podStartE2EDuration="1.924274685s" totalImagesPullingTime="175.188058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:26:37.923570157 +0000 UTC m=+664.244668296" watchObservedRunningTime="2026-09-19 04:26:37.924274685 +0000 UTC m=+664.245372832" Sep 19 04:26:43.936577 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:43.936543 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd"] Sep 19 04:26:43.940100 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:43.940084 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:43.945145 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:43.945127 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 19 04:26:43.955094 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:43.955068 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd"] Sep 19 04:26:44.047842 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.047807 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.048043 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.047850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.048043 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.048011 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.048184 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.048060 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16217be-a67c-4318-96b6-4942f3d9d2f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.048184 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.048111 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.048184 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.048141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfclp\" (UniqueName: \"kubernetes.io/projected/b16217be-a67c-4318-96b6-4942f3d9d2f4-kube-api-access-sfclp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.051452 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.051415 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:26:44.149207 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.149143 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.149404 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.149220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.149404 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.149308 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.149404 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.149342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16217be-a67c-4318-96b6-4942f3d9d2f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.149404 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.149377 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.149626 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.149477 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfclp\" (UniqueName: \"kubernetes.io/projected/b16217be-a67c-4318-96b6-4942f3d9d2f4-kube-api-access-sfclp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.149685 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.149630 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.149742 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.149688 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.149742 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.149733 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.151783 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.151761 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16217be-a67c-4318-96b6-4942f3d9d2f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.151895 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.151870 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16217be-a67c-4318-96b6-4942f3d9d2f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.158359 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.158339 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfclp\" (UniqueName: \"kubernetes.io/projected/b16217be-a67c-4318-96b6-4942f3d9d2f4-kube-api-access-sfclp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xtbd\" (UID: \"b16217be-a67c-4318-96b6-4942f3d9d2f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.250434 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.250335 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:44.527060 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.527036 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd"] Sep 19 04:26:44.528245 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:26:44.528216 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb16217be_a67c_4318_96b6_4942f3d9d2f4.slice/crio-18e28accde187137b9f71dd178328a294408c55c286b067c56291afb06562f50 WatchSource:0}: Error finding container 18e28accde187137b9f71dd178328a294408c55c286b067c56291afb06562f50: Status 404 returned error can't find the container with id 18e28accde187137b9f71dd178328a294408c55c286b067c56291afb06562f50 Sep 19 04:26:44.932235 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.932108 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" event={"ID":"b16217be-a67c-4318-96b6-4942f3d9d2f4","Type":"ContainerStarted","Data":"0e717978af96df30d7989918a1f79c6b2df24ac73d540c3ace8a1fbf5e2aaab4"} Sep 19 04:26:44.932235 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.932184 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" event={"ID":"b16217be-a67c-4318-96b6-4942f3d9d2f4","Type":"ContainerStarted","Data":"18e28accde187137b9f71dd178328a294408c55c286b067c56291afb06562f50"} Sep 19 04:26:44.961449 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:44.961393 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" podStartSLOduration=1.782218531 podStartE2EDuration="1.961376176s" totalImagesPullingTime="179.157645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:26:44.959483068 +0000 UTC m=+671.280581213" watchObservedRunningTime="2026-09-19 04:26:44.961376176 +0000 UTC m=+671.282474321" Sep 19 04:26:46.723115 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:46.723074 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:46.723629 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:46.723171 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:46.735869 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:46.735831 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:46.951929 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:46.951899 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr" Sep 19 04:26:54.251262 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:54.251222 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:54.251736 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:54.251287 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:54.263917 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:54.263888 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:26:54.986741 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:26:54.986711 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xtbd" Sep 19 04:27:00.323285 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.323254 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt"] Sep 19 04:27:00.329303 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.329274 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.332292 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.332274 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 19 04:27:00.341895 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.341872 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt"] Sep 19 04:27:00.357292 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.357247 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:27:00.504310 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.504268 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.504500 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.504317 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60735b5-b631-4786-8ad7-d8a52547e7a4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.504500 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.504379 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.504500 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.504437 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.504500 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.504466 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9xq7\" (UniqueName: \"kubernetes.io/projected/c60735b5-b631-4786-8ad7-d8a52547e7a4-kube-api-access-n9xq7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.504500 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.504493 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.605395 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.605302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.605395 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.605341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60735b5-b631-4786-8ad7-d8a52547e7a4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.605395 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.605382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.605666 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.605437 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.605666 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.605459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9xq7\" (UniqueName: \"kubernetes.io/projected/c60735b5-b631-4786-8ad7-d8a52547e7a4-kube-api-access-n9xq7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.605666 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.605488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.605827 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.605787 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.605827 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.605810 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.605939 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.605910 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.607809 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.607788 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60735b5-b631-4786-8ad7-d8a52547e7a4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.607975 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.607957 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60735b5-b631-4786-8ad7-d8a52547e7a4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.614485 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.614462 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9xq7\" (UniqueName: \"kubernetes.io/projected/c60735b5-b631-4786-8ad7-d8a52547e7a4-kube-api-access-n9xq7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt\" (UID: \"c60735b5-b631-4786-8ad7-d8a52547e7a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.639860 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.639835 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:00.915299 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:00.915271 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt"] Sep 19 04:27:00.918244 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:27:00.918205 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60735b5_b631_4786_8ad7_d8a52547e7a4.slice/crio-b746200412ec4c000caa0e31179b03da41583f9e5ac5ec00e68a4a44d4e20880 WatchSource:0}: Error finding container b746200412ec4c000caa0e31179b03da41583f9e5ac5ec00e68a4a44d4e20880: Status 404 returned error can't find the container with id b746200412ec4c000caa0e31179b03da41583f9e5ac5ec00e68a4a44d4e20880 Sep 19 04:27:01.004730 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:01.004686 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" event={"ID":"c60735b5-b631-4786-8ad7-d8a52547e7a4","Type":"ContainerStarted","Data":"b746200412ec4c000caa0e31179b03da41583f9e5ac5ec00e68a4a44d4e20880"} Sep 19 04:27:02.010394 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:02.010353 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" event={"ID":"c60735b5-b631-4786-8ad7-d8a52547e7a4","Type":"ContainerStarted","Data":"80f6cf79dd203d4798d03b283f9ea2b44c176368143ebbc49ed03c9d90ff2e3d"} Sep 19 04:27:02.036083 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:02.036031 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" podStartSLOduration=1.876069612 podStartE2EDuration="2.036015138s" totalImagesPullingTime="159.945526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 04:27:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:27:02.03450141 +0000 UTC m=+688.355599557" watchObservedRunningTime="2026-09-19 04:27:02.036015138 +0000 UTC m=+688.357113302" Sep 19 04:27:09.448001 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:09.447959 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:27:10.640700 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:10.640661 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:10.641113 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:10.640755 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:10.653405 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:10.653372 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:11.064620 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:11.064591 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt" Sep 19 04:27:18.165950 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:27:18.165843 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:06.587480 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:06.587441 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:06.683857 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:06.683818 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:09.557771 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:09.557729 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:14.250866 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:14.250753 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:14.353588 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:14.353545 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:47.183786 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:47.183703 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:56.819947 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:56.819910 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw"] Sep 19 04:28:56.820411 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:56.820107 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" podUID="99ce8be0-22ad-4343-99b1-187ffc142a90" containerName="payload-pre-processing" containerID="cri-o://bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5" gracePeriod=30 Sep 19 04:28:56.866473 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:56.866433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:56.970138 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:56.970099 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:57.066220 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.066193 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:28:57.205235 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.205129 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ce8be0-22ad-4343-99b1-187ffc142a90-plugins-config-volume\") pod \"99ce8be0-22ad-4343-99b1-187ffc142a90\" (UID: \"99ce8be0-22ad-4343-99b1-187ffc142a90\") " Sep 19 04:28:57.205413 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.205240 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckq5r\" (UniqueName: \"kubernetes.io/projected/99ce8be0-22ad-4343-99b1-187ffc142a90-kube-api-access-ckq5r\") pod \"99ce8be0-22ad-4343-99b1-187ffc142a90\" (UID: \"99ce8be0-22ad-4343-99b1-187ffc142a90\") " Sep 19 04:28:57.205518 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.205493 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99ce8be0-22ad-4343-99b1-187ffc142a90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "99ce8be0-22ad-4343-99b1-187ffc142a90" (UID: "99ce8be0-22ad-4343-99b1-187ffc142a90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 04:28:57.207518 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.207492 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99ce8be0-22ad-4343-99b1-187ffc142a90-kube-api-access-ckq5r" (OuterVolumeSpecName: "kube-api-access-ckq5r") pod "99ce8be0-22ad-4343-99b1-187ffc142a90" (UID: "99ce8be0-22ad-4343-99b1-187ffc142a90"). InnerVolumeSpecName "kube-api-access-ckq5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 04:28:57.306085 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.306054 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckq5r\" (UniqueName: \"kubernetes.io/projected/99ce8be0-22ad-4343-99b1-187ffc142a90-kube-api-access-ckq5r\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:28:57.306085 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.306081 2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ce8be0-22ad-4343-99b1-187ffc142a90-plugins-config-volume\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\"" Sep 19 04:28:57.545073 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.545042 2577 generic.go:356] "Generic (PLEG): container finished" podID="99ce8be0-22ad-4343-99b1-187ffc142a90" containerID="bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5" exitCode=0 Sep 19 04:28:57.545292 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.545092 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" Sep 19 04:28:57.545292 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.545122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" event={"ID":"99ce8be0-22ad-4343-99b1-187ffc142a90","Type":"ContainerDied","Data":"bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5"} Sep 19 04:28:57.545292 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.545191 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw" event={"ID":"99ce8be0-22ad-4343-99b1-187ffc142a90","Type":"ContainerDied","Data":"aa6fbda6a350b1b67b2b6709ef501406d20ac720f83540a9b0c566c52c0fd2ee"} Sep 19 04:28:57.545292 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.545216 2577 scope.go:117] "RemoveContainer" containerID="bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5" Sep 19 04:28:57.555214 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.555191 2577 scope.go:117] "RemoveContainer" containerID="bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5" Sep 19 04:28:57.555529 ip-10-0-132-168 kubenswrapper[2577]: E0919 04:28:57.555500 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5\": container with ID starting with bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5 not found: ID does not exist" containerID="bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5" Sep 19 04:28:57.555592 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.555544 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5"} err="failed to get container status \"bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5\": rpc error: code = NotFound desc = could not find container \"bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5\": container with ID starting with bd67ee886c2b6f0096a2d203b64cc1db2100ce6d9a69ed66b934e2ef3d113cd5 not found: ID does not exist" Sep 19 04:28:57.579928 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.579893 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw"] Sep 19 04:28:57.583322 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.583299 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9v8gw"] Sep 19 04:28:57.599665 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.599639 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:57.704090 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.704035 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:28:57.893948 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:28:57.893873 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99ce8be0-22ad-4343-99b1-187ffc142a90" path="/var/lib/kubelet/pods/99ce8be0-22ad-4343-99b1-187ffc142a90/volumes" Sep 19 04:29:04.024231 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:04.024204 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8fdcb89-cwgg9_b825d08c-6b41-4bf0-9b48-345ab97d8eea/copy-manifests/0.log" Sep 19 04:29:04.047187 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:04.044768 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8fdcb89-cwgg9_b825d08c-6b41-4bf0-9b48-345ab97d8eea/manager/0.log" Sep 19 04:29:05.920016 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:05.919992 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-tqqpb_ef877b8d-5f04-4c5e-8542-5104dd3e83ab/kuadrant-console-plugin/0.log" Sep 19 04:29:06.043470 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:06.043437 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-vtrkj_b791d0e1-1512-423b-bfa8-d9b94bc9862e/manager/0.log" Sep 19 04:29:07.012104 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:07.012058 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77df8579b6-fd2ss_b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5/kube-auth-proxy/0.log" Sep 19 04:29:07.479022 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:07.478994 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9c66fb948-4h58s_133f6fcb-808b-4ae3-a06d-822efa9184a5/router/0.log" Sep 19 04:29:07.943181 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:07.943140 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5pxwr_30446db5-4131-4338-a2ec-1dee107d99bc/main/0.log" Sep 19 04:29:08.061635 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:08.061605 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr_4fc54592-cb93-45bc-83d5-a9619cce5a0f/main/0.log" Sep 19 04:29:08.177868 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:08.177843 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8xtbd_b16217be-a67c-4318-96b6-4942f3d9d2f4/main/0.log" Sep 19 04:29:08.292032 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:08.292000 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt_c60735b5-b631-4786-8ad7-d8a52547e7a4/main/0.log" Sep 19 04:29:15.898004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:15.897973 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8fdcb89-cwgg9_b825d08c-6b41-4bf0-9b48-345ab97d8eea/copy-manifests/0.log" Sep 19 04:29:15.923424 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:15.923394 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8fdcb89-cwgg9_b825d08c-6b41-4bf0-9b48-345ab97d8eea/manager/0.log" Sep 19 04:29:17.807202 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:17.807170 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-tqqpb_ef877b8d-5f04-4c5e-8542-5104dd3e83ab/kuadrant-console-plugin/0.log" Sep 19 04:29:17.926615 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:17.926575 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-vtrkj_b791d0e1-1512-423b-bfa8-d9b94bc9862e/manager/0.log" Sep 19 04:29:18.840139 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:18.840104 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77df8579b6-fd2ss_b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5/kube-auth-proxy/0.log" Sep 19 04:29:19.293530 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:19.293499 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9c66fb948-4h58s_133f6fcb-808b-4ae3-a06d-822efa9184a5/router/0.log" Sep 19 04:29:19.764258 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:19.764167 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5pxwr_30446db5-4131-4338-a2ec-1dee107d99bc/main/0.log" Sep 19 04:29:19.888800 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:19.888767 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-zbpkr_4fc54592-cb93-45bc-83d5-a9619cce5a0f/main/0.log" Sep 19 04:29:20.003070 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:20.003008 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8xtbd_b16217be-a67c-4318-96b6-4942f3d9d2f4/main/0.log" Sep 19 04:29:20.114053 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:20.114024 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmdlrt_c60735b5-b631-4786-8ad7-d8a52547e7a4/main/0.log" Sep 19 04:29:35.939543 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:35.939511 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:29:36.042798 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:36.042764 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" limit=1024 Sep 19 04:29:38.876027 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:38.876001 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8lnq4_4eea2ebc-5430-4f00-9ed7-492efe045fb0/global-pull-secret-syncer/0.log" Sep 19 04:29:39.043133 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:39.043090 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cnmp2_7dfafbf3-d464-424a-96c9-ef104e8ede27/konnectivity-agent/0.log" Sep 19 04:29:39.146419 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:39.146306 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-168.ec2.internal_aba6b42d05a06c053c050c26af669cbb/haproxy/0.log" Sep 19 04:29:43.829792 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:43.829762 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-tqqpb_ef877b8d-5f04-4c5e-8542-5104dd3e83ab/kuadrant-console-plugin/0.log" Sep 19 04:29:43.882508 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:43.882475 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-vtrkj_b791d0e1-1512-423b-bfa8-d9b94bc9862e/manager/0.log" Sep 19 04:29:45.565329 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:45.565291 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-k89dx_cc4411b4-df39-4284-bba8-a14812163819/kube-state-metrics/0.log" Sep 19 04:29:45.585556 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:45.585526 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-k89dx_cc4411b4-df39-4284-bba8-a14812163819/kube-rbac-proxy-main/0.log" Sep 19 04:29:45.607386 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:45.607339 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-k89dx_cc4411b4-df39-4284-bba8-a14812163819/kube-rbac-proxy-self/0.log" Sep 19 04:29:45.642231 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:45.642206 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6dfd8fc58b-tq5h5_c61fb3ac-0fc4-487a-9fe8-d3f74a660f81/metrics-server/0.log" Sep 19 04:29:45.720391 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:45.720360 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9cxmb_249ddd11-73ec-42f8-8212-c7d7110de056/node-exporter/0.log" Sep 19 04:29:45.755625 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:45.755600 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9cxmb_249ddd11-73ec-42f8-8212-c7d7110de056/kube-rbac-proxy/0.log" Sep 19 04:29:45.784404 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:45.784383 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9cxmb_249ddd11-73ec-42f8-8212-c7d7110de056/init-textfile/0.log" Sep 19 04:29:46.025464 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.025434 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-z8jkv_805b7bf9-b662-400b-a35a-d52ff284ca23/kube-rbac-proxy-main/0.log" Sep 19 04:29:46.052114 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.052085 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-z8jkv_805b7bf9-b662-400b-a35a-d52ff284ca23/kube-rbac-proxy-self/0.log" Sep 19 04:29:46.076374 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.076346 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-z8jkv_805b7bf9-b662-400b-a35a-d52ff284ca23/openshift-state-metrics/0.log" Sep 19 04:29:46.115284 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.115256 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_be47a7bd-f90d-4a04-9be2-e84845dd0bbf/prometheus/0.log" Sep 19 04:29:46.139584 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.139560 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_be47a7bd-f90d-4a04-9be2-e84845dd0bbf/config-reloader/0.log" Sep 19 04:29:46.181212 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.181184 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_be47a7bd-f90d-4a04-9be2-e84845dd0bbf/thanos-sidecar/0.log" Sep 19 04:29:46.232916 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.232885 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_be47a7bd-f90d-4a04-9be2-e84845dd0bbf/kube-rbac-proxy-web/0.log" Sep 19 04:29:46.280975 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.280889 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_be47a7bd-f90d-4a04-9be2-e84845dd0bbf/kube-rbac-proxy/0.log" Sep 19 04:29:46.330863 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.330840 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_be47a7bd-f90d-4a04-9be2-e84845dd0bbf/kube-rbac-proxy-thanos/0.log" Sep 19 04:29:46.372775 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.372751 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_be47a7bd-f90d-4a04-9be2-e84845dd0bbf/init-config-reloader/0.log" Sep 19 04:29:46.417832 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.417807 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-knh8x_610ed2d1-470b-4c8d-973f-688c6de8053e/prometheus-operator/0.log" Sep 19 04:29:46.459059 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:46.459027 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-knh8x_610ed2d1-470b-4c8d-973f-688c6de8053e/kube-rbac-proxy/0.log" Sep 19 04:29:48.320237 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.320175 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-2j44x_db9e87e7-0480-4019-9416-827502badf63/networking-console-plugin/0.log" Sep 19 04:29:48.596018 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.595933 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5"] Sep 19 04:29:48.596333 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.596319 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99ce8be0-22ad-4343-99b1-187ffc142a90" containerName="payload-pre-processing" Sep 19 04:29:48.596390 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.596335 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99ce8be0-22ad-4343-99b1-187ffc142a90" containerName="payload-pre-processing" Sep 19 04:29:48.596428 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.596399 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99ce8be0-22ad-4343-99b1-187ffc142a90" containerName="payload-pre-processing" Sep 19 04:29:48.599330 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.599313 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.602190 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.602171 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wpk7m\"/\"openshift-service-ca.crt\"" Sep 19 04:29:48.602457 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.602438 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wpk7m\"/\"default-dockercfg-kvqhs\"" Sep 19 04:29:48.602986 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.602970 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wpk7m\"/\"kube-root-ca.crt\"" Sep 19 04:29:48.611298 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.611276 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5"] Sep 19 04:29:48.674904 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.674866 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-proc\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.675077 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.674933 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-sys\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.675077 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.674972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grkz9\" (UniqueName: \"kubernetes.io/projected/ff5ccb03-b361-4674-8465-1fcd254a6ebc-kube-api-access-grkz9\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.675077 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.674993 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-lib-modules\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.675077 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.675056 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-podres\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.776531 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.776492 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkz9\" (UniqueName: \"kubernetes.io/projected/ff5ccb03-b361-4674-8465-1fcd254a6ebc-kube-api-access-grkz9\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.776531 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.776536 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-lib-modules\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.776781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.776561 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-podres\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.776781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.776600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-proc\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.776781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.776656 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-sys\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.776781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.776725 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-proc\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.776781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.776734 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-podres\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.776781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.776748 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-sys\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.776781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.776734 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff5ccb03-b361-4674-8465-1fcd254a6ebc-lib-modules\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.785762 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.785737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkz9\" (UniqueName: \"kubernetes.io/projected/ff5ccb03-b361-4674-8465-1fcd254a6ebc-kube-api-access-grkz9\") pod \"perf-node-gather-daemonset-qc8f5\" (UID: \"ff5ccb03-b361-4674-8465-1fcd254a6ebc\") " pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:48.909781 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:48.909691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:49.183438 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:49.183410 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5"] Sep 19 04:29:49.185886 ip-10-0-132-168 kubenswrapper[2577]: W0919 04:29:49.185859 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podff5ccb03_b361_4674_8465_1fcd254a6ebc.slice/crio-f799aa6a3160e4af3c0e47abd0387a2276f3a2e6d56bed96347e152065d15178 WatchSource:0}: Error finding container f799aa6a3160e4af3c0e47abd0387a2276f3a2e6d56bed96347e152065d15178: Status 404 returned error can't find the container with id f799aa6a3160e4af3c0e47abd0387a2276f3a2e6d56bed96347e152065d15178 Sep 19 04:29:49.589234 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:49.589197 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64cfdb4dc9-scgkc_80b4d933-4265-4f0f-8086-965ffcb6964b/console/0.log" Sep 19 04:29:49.787425 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:49.787384 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" event={"ID":"ff5ccb03-b361-4674-8465-1fcd254a6ebc","Type":"ContainerStarted","Data":"9b1e9446961951c56956f9c1184fc32e4988b1a182c16da3c21a7aa4cd96b03f"} Sep 19 04:29:49.787611 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:49.787451 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" event={"ID":"ff5ccb03-b361-4674-8465-1fcd254a6ebc","Type":"ContainerStarted","Data":"f799aa6a3160e4af3c0e47abd0387a2276f3a2e6d56bed96347e152065d15178"} Sep 19 04:29:49.787611 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:49.787481 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:49.808275 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:49.808232 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" podStartSLOduration=1.808216263 podStartE2EDuration="1.808216263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 04:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 04:29:49.806352873 +0000 UTC m=+856.127451019" watchObservedRunningTime="2026-09-19 04:29:49.808216263 +0000 UTC m=+856.129314408" Sep 19 04:29:50.316166 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:50.316130 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-7dvk6_b969e868-5636-4a8c-af91-d8520bb37efc/volume-data-source-validator/0.log" Sep 19 04:29:51.563682 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:51.563636 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xzvch_58784ca9-33f1-4d36-aefc-455a3568bf85/dns/0.log" Sep 19 04:29:51.590136 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:51.590097 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xzvch_58784ca9-33f1-4d36-aefc-455a3568bf85/kube-rbac-proxy/0.log" Sep 19 04:29:51.649113 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:51.649081 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rjjrv_ac630975-bfc0-46ba-9195-78394cc767ea/dns-node-resolver/0.log" Sep 19 04:29:52.341569 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:52.341521 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-645z6_c3d3a01e-11a7-42ec-9ec0-72d1c5b002dd/node-ca/0.log" Sep 19 04:29:53.569410 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:53.569380 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77df8579b6-fd2ss_b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5/kube-auth-proxy/0.log" Sep 19 04:29:53.704327 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:53.704288 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9c66fb948-4h58s_133f6fcb-808b-4ae3-a06d-822efa9184a5/router/0.log" Sep 19 04:29:54.439004 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:54.438968 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gcq92_509b6215-7037-419b-8ca4-65d1c028e244/serve-healthcheck-canary/0.log" Sep 19 04:29:55.024783 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:55.024751 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-p5dtn_0f534b12-a447-453d-aa10-9118ac1b6d53/insights-operator/0.log" Sep 19 04:29:55.025183 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:55.024802 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-p5dtn_0f534b12-a447-453d-aa10-9118ac1b6d53/insights-operator/1.log" Sep 19 04:29:55.127396 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:55.127365 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bm85n_c2a4613e-bd9e-4c5b-96e6-224b6615fad6/kube-rbac-proxy/0.log" Sep 19 04:29:55.150028 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:55.149994 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bm85n_c2a4613e-bd9e-4c5b-96e6-224b6615fad6/exporter/0.log" Sep 19 04:29:55.172867 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:55.172832 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bm85n_c2a4613e-bd9e-4c5b-96e6-224b6615fad6/extractor/0.log" Sep 19 04:29:55.802443 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:55.802417 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wpk7m/perf-node-gather-daemonset-qc8f5" Sep 19 04:29:57.874782 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:57.874732 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8fdcb89-cwgg9_b825d08c-6b41-4bf0-9b48-345ab97d8eea/manager/0.log" Sep 19 04:29:57.955010 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:57.954983 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78d8fdcb89-cwgg9_b825d08c-6b41-4bf0-9b48-345ab97d8eea/copy-manifests/0.log" Sep 19 04:29:59.164439 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:59.164409 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77df8579b6-fd2ss_b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5/kube-auth-proxy/0.log" Sep 19 04:29:59.245556 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:59.245525 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9c66fb948-4h58s_133f6fcb-808b-4ae3-a06d-822efa9184a5/router/0.log" Sep 19 04:29:59.386651 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:29:59.386613 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7755db756b-trxfn_6a04db80-840b-4a5f-9d2a-e145a34815f4/manager/0.log" Sep 19 04:30:00.488434 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:30:00.488353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77df8579b6-fd2ss_b6ab50b8-7a19-4c1a-8c3b-f3f89e5bb5e5/kube-auth-proxy/0.log" Sep 19 04:30:00.565215 ip-10-0-132-168 kubenswrapper[2577]: I0919 04:30:00.565186 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9c66fb948-4h58s_133f6fcb-808b-4ae3-a06d-822efa9184a5/router/0.log"