Sep 10 12:06:18.164079 ip-10-0-141-9 systemd[1]: Starting Kubernetes Kubelet... Sep 10 12:06:18.517472 ip-10-0-141-9 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:06:18.517472 ip-10-0-141-9 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 12:06:18.517472 ip-10-0-141-9 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:06:18.517472 ip-10-0-141-9 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 12:06:18.517472 ip-10-0-141-9 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:06:18.519718 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.519629 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 12:06:18.525070 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525051 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:06:18.525070 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525069 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525075 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525079 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525083 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525086 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525089 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525092 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525095 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525097 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525100 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525103 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525105 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525107 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525110 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525112 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525115 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525118 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525120 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525123 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:06:18.525142 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525125 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525140 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525143 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525145 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525148 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525151 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525154 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525157 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525159 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525162 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525164 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525166 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525169 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525172 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525174 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525177 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525179 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525182 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525184 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525187 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:06:18.525700 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525189 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525191 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525194 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525196 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525199 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525201 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525204 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525206 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525208 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525212 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525214 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525217 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525219 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525222 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525227 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525232 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525237 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525240 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:06:18.526202 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525243 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525246 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525249 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525251 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525254 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525257 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525259 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525262 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525264 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525266 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525269 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525271 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525273 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525276 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525278 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525280 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525283 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525285 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525287 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525290 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:06:18.526649 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525293 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525296 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525298 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525301 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525304 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525306 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525309 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525311 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525314 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525318 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525320 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525323 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525325 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525328 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525330 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525333 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525335 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525338 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525340 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525343 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:06:18.527146 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.525345 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526072 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526079 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526082 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526085 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526088 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526091 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526093 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526096 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526098 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526101 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526103 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526107 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526109 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526111 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526114 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526117 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526119 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526122 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526124 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:06:18.527630 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526128 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526131 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526134 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526136 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526139 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526141 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526144 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526146 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526148 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526151 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526154 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526156 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526160 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526164 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526167 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526170 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526172 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526175 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526178 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:06:18.528143 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526181 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526183 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526186 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526188 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526191 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526194 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526196 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526199 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526201 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526204 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526206 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526209 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526212 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526215 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526218 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526221 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526224 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526226 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526229 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:06:18.528611 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526232 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526234 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526237 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526239 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526242 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526245 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526247 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526253 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526256 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526259 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526261 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526264 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526266 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526269 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526271 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526274 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526276 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526279 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526281 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526284 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:06:18.529087 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526287 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526289 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526291 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526294 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526296 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526299 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526302 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526304 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526307 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526309 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526312 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526314 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526317 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526319 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526321 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526324 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526328 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526331 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526333 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:06:18.529626 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526336 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526339 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526342 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526725 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526731 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526734 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526737 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526740 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526743 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526746 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526748 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526751 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526754 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526757 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526759 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526762 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526764 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526783 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526788 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:06:18.530137 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526792 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526795 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526798 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526801 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526804 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526807 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526809 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526812 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526814 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526817 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526819 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526822 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526824 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526826 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526829 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526831 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526834 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526836 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526839 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526841 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:06:18.530744 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526843 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526846 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526849 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526852 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526855 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526858 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526860 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526863 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526865 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526867 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526870 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526873 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526876 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526878 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526881 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526883 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526885 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526888 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526891 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526893 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:06:18.531259 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526896 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526898 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526900 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526903 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526905 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526908 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526910 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526912 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526915 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526917 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526920 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526922 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526925 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526927 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526930 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526932 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526935 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526937 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526940 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526942 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:06:18.532037 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526944 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526947 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526949 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526951 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526957 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526959 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526962 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526964 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526966 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526969 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526972 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526974 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526976 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526979 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526981 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526983 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526986 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526988 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526990 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526994 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:06:18.532553 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.526997 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527000 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527003 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527087 2583 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527123 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527130 2583 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527134 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527139 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527142 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527147 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527151 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527154 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527157 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527161 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527164 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527167 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527171 2583 flags.go:64] FLAG: --cgroup-root="" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527174 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527177 2583 flags.go:64] FLAG: --client-ca-file="" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527180 2583 flags.go:64] FLAG: --cloud-provider="external" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527183 2583 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527187 2583 flags.go:64] FLAG: --cluster-domain="" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527191 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527194 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 12:06:18.533072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527197 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527201 2583 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527204 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527207 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527210 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527213 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527216 2583 flags.go:64] FLAG: --contention-profiling="false" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527219 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527222 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527225 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527228 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527232 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527235 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527237 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527240 2583 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527243 2583 flags.go:64] FLAG: --enable-server="true" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527248 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527253 2583 flags.go:64] FLAG: --event-burst="100" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527256 2583 flags.go:64] FLAG: --event-qps="50" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527259 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527262 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527265 2583 flags.go:64] FLAG: --eviction-hard="" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527269 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527271 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527274 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 12:06:18.533679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527277 2583 flags.go:64] FLAG: --eviction-soft="" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527280 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527283 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527286 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527289 2583 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527292 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527295 2583 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527299 2583 flags.go:64] FLAG: --feature-gates="" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527303 2583 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527305 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527308 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527311 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527314 2583 flags.go:64] FLAG: --healthz-port="10248" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527318 2583 flags.go:64] FLAG: --help="false" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527320 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527324 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527327 2583 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527330 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527333 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527337 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527339 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527342 2583 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527345 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 12:06:18.534313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527349 2583 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527352 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527355 2583 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527358 2583 flags.go:64] FLAG: --kube-reserved="" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527361 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527364 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527367 2583 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527369 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527372 2583 flags.go:64] FLAG: --lock-file="" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527375 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527378 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527381 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527386 2583 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527389 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527392 2583 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527395 2583 flags.go:64] FLAG: --logging-format="text" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527397 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527401 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527404 2583 flags.go:64] FLAG: --manifest-url="" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527407 2583 flags.go:64] FLAG: --manifest-url-header="" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527412 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527415 2583 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527419 2583 flags.go:64] FLAG: --max-pods="110" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527422 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527425 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 12:06:18.534920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527427 2583 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527430 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527433 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527436 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527439 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527446 2583 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527449 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527453 2583 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527456 2583 flags.go:64] FLAG: --pod-cidr="" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527459 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527465 2583 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527468 2583 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527471 2583 flags.go:64] FLAG: --pods-per-core="0" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527474 2583 flags.go:64] FLAG: --port="10250" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527477 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527479 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ed25abb972762035" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527482 2583 flags.go:64] FLAG: --qos-reserved="" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527485 2583 flags.go:64] FLAG: --read-only-port="10255" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527488 2583 flags.go:64] FLAG: --register-node="true" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527491 2583 flags.go:64] FLAG: --register-with-taints="" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527494 2583 flags.go:64] FLAG: --registry-burst="10" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527497 2583 flags.go:64] FLAG: --registry-qps="5" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527500 2583 flags.go:64] FLAG: --reserved-cpus="" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527503 2583 flags.go:64] FLAG: --reserved-memory="" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527506 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 12:06:18.535541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527510 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527513 2583 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527516 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527518 2583 flags.go:64] FLAG: --runonce="false" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527521 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527524 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527527 2583 flags.go:64] FLAG: --seccomp-default="false" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527530 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527532 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527535 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527538 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527542 2583 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527544 2583 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527547 2583 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527550 2583 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527554 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527557 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527560 2583 flags.go:64] FLAG: --system-cgroups="" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527563 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527570 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527573 2583 flags.go:64] FLAG: --tls-cert-file="" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527576 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527581 2583 flags.go:64] FLAG: --tls-min-version="" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527584 2583 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527588 2583 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 12:06:18.536204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527591 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527594 2583 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527597 2583 flags.go:64] FLAG: --v="2" Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527601 2583 flags.go:64] FLAG: --version="false" Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527605 2583 flags.go:64] FLAG: --vmodule="" Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527608 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.527612 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527724 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527729 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527732 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527736 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527739 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527742 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527745 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527747 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527749 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527752 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527754 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527756 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527759 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527761 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:06:18.536859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527764 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527783 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527785 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527788 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527791 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527793 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527796 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527799 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527803 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527805 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527808 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527810 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527813 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527815 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527817 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527820 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527823 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527825 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527828 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527831 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:06:18.537394 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527837 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527840 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527842 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527845 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527847 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527850 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527852 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527855 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527857 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527860 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527862 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527864 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527867 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527871 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527873 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527875 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527878 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527880 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527883 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:06:18.537919 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527885 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527889 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527891 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527894 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527896 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527899 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527901 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527904 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527906 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527910 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527914 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527917 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527920 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527922 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527926 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527928 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527931 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527933 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527935 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:06:18.538383 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527938 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527942 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527945 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527948 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527951 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527953 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527956 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527961 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527963 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527966 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527969 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527972 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527975 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527978 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527981 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527984 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527986 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527989 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527992 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:06:18.538954 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527994 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527996 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.527999 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.528001 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.528004 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.528006 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.528009 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.528011 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.528017 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.537801 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.537936 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.537957 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.537963 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538022 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538027 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:06:18.539424 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538030 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538034 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538037 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538040 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538043 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538046 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538048 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538051 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538053 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538056 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538058 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538061 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538064 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538066 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538069 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538071 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538074 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538077 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538080 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:06:18.539852 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538083 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538085 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538088 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538090 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538093 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538098 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538102 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538105 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538109 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538111 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538114 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538117 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538119 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538122 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538124 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538127 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538129 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538132 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538134 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:06:18.540316 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538138 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538141 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538145 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538147 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538150 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538153 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538157 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538160 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538162 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538165 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538167 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538170 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538173 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538175 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538178 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538180 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538183 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538186 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538188 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538191 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:06:18.540790 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538193 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538197 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538201 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538204 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538206 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538210 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538212 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538215 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538217 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538220 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538222 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538225 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538227 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538230 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538232 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538235 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538237 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538240 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538242 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538245 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:06:18.541337 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538247 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538250 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538252 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538255 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538257 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538260 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538263 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538265 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538267 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538270 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538272 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538275 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538277 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538279 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538282 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538284 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538286 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538289 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:06:18.541859 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538291 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.538296 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538408 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538413 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538416 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538419 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538422 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538425 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538428 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538430 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538434 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538436 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538439 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538441 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538444 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:06:18.542348 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538447 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538449 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538452 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538454 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538457 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538460 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538462 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538465 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538467 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538470 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538472 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538474 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538477 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538479 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538482 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538485 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538487 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538490 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538492 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538494 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:06:18.542713 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538497 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538499 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538502 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538505 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538507 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538509 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538512 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538515 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538518 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538520 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538522 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538525 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538529 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538531 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538534 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538536 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538539 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538541 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538544 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:06:18.543223 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538546 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538549 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538551 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538554 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538556 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538558 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538561 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538563 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538566 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538569 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538571 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538573 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538576 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538578 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538581 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538584 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538587 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538590 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538592 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:06:18.543688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538595 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538597 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538600 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538602 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538604 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538607 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538610 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538612 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538615 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538617 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538620 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538625 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538629 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538632 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538655 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538659 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538663 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538666 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538669 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538671 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:06:18.544184 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538674 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538676 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538678 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538681 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538685 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538688 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538691 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:18.538694 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.538698 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.538843 2583 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.540991 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.541800 2583 server.go:1019] "Starting client certificate rotation" Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.541902 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 12:06:18.544682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.541936 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 12:06:18.545022 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.544117 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 12:06:18.545265 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.545244 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 12:06:18.556904 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.556885 2583 log.go:25] "Validated CRI v1 runtime API" Sep 10 12:06:18.562509 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.562491 2583 log.go:25] "Validated CRI v1 image API" Sep 10 12:06:18.563374 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.563358 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 12:06:18.566147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.566129 2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b3622ddf-d65b-482c-bb64-5c5e92d58679:/dev/nvme0n1p4 f7332718-d0d2-49ab-8095-a640a7930432:/dev/nvme0n1p3] Sep 10 12:06:18.566200 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.566147 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 12:06:18.572153 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.572035 2583 manager.go:216] Machine: {Timestamp:2026-09-10 12:06:18.570902919 +0000 UTC m=+0.297807458 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093249 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2349cba8be6f7a6c8654d38bcb7f7a SystemUUID:ec2349cb-a8be-6f7a-6c86-54d38bcb7f7a BootID:f1811220-f76d-40e4-88ed-9819a59a509c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1c:b6:3f:c1:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1c:b6:3f:c1:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:f3:7e:28:6f:0a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 12:06:18.572153 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.572150 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 12:06:18.572282 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.572270 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 12:06:18.575295 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.575268 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 12:06:18.575442 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.575299 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-9.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 12:06:18.575798 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.575759 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 12:06:18.576583 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.576571 2583 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 12:06:18.576616 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.576587 2583 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 12:06:18.576616 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.576596 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 12:06:18.577243 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.577233 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 12:06:18.578745 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.578734 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 12:06:18.583367 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.583353 2583 state_mem.go:36] "Initialized new in-memory state store" Sep 10 12:06:18.583669 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.583657 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 12:06:18.583700 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.583691 2583 kubelet.go:486] "Attempting to sync node with API server" Sep 10 12:06:18.583727 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.583700 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 12:06:18.583727 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.583716 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 12:06:18.583811 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.583727 2583 kubelet.go:398] "Adding apiserver pod source" Sep 10 12:06:18.583811 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.583737 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 12:06:18.584143 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.584118 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 12:06:18.584143 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.584136 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 12:06:18.586441 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.586426 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 12:06:18.588964 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.588950 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 12:06:18.589026 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.588981 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 12:06:18.589161 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589151 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 12:06:18.589190 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589166 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 12:06:18.589190 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589172 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 12:06:18.589190 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589177 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 12:06:18.589190 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589186 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 12:06:18.589190 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589192 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 12:06:18.589651 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589642 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 12:06:18.589651 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589652 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 12:06:18.589704 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589658 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 12:06:18.589704 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589663 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 12:06:18.589704 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589680 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 12:06:18.589704 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.589687 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 12:06:18.590446 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.590435 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 12:06:18.590482 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.590448 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 12:06:18.594182 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.594168 2583 server.go:1268] "Started kubelet" Sep 10 12:06:18.594293 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.594250 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 12:06:18.594347 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.594281 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 12:06:18.594436 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.594418 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-9.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 12:06:18.594724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.594323 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 12:06:18.595051 ip-10-0-141-9 systemd[1]: Started Kubernetes Kubelet. Sep 10 12:06:18.595176 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.595126 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-9.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 12:06:18.595176 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.595125 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 12:06:18.596698 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.596684 2583 server.go:310] "Adding debug handlers to kubelet server" Sep 10 12:06:18.598938 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.598921 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 12:06:18.600735 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.600716 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 12:06:18.601754 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.600875 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-9.ec2.internal.18d3f433665d9b5f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-9.ec2.internal,UID:ip-10-0-141-9.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-9.ec2.internal,},FirstTimestamp:2026-09-10 12:06:18.594138975 +0000 UTC m=+0.321043566,LastTimestamp:2026-09-10 12:06:18.594138975 +0000 UTC m=+0.321043566,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-9.ec2.internal,}" Sep 10 12:06:18.602982 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.602963 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 12:06:18.603549 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.603534 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 12:06:18.603649 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.603632 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:18.604551 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.604530 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 12:06:18.604620 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.604555 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 12:06:18.604723 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.604706 2583 reconstruct.go:97] "Volume reconstruction finished" Sep 10 12:06:18.604723 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.604713 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 12:06:18.604884 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.604719 2583 reconciler.go:29] "Reconciler: start to sync state" Sep 10 12:06:18.605952 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.605924 2583 factory.go:55] Registering systemd factory Sep 10 12:06:18.605952 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.605948 2583 factory.go:223] Registration of the systemd container factory successfully Sep 10 12:06:18.606220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.606207 2583 factory.go:153] Registering CRI-O factory Sep 10 12:06:18.606285 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.606222 2583 factory.go:223] Registration of the crio container factory successfully Sep 10 12:06:18.606285 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.606270 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 12:06:18.606373 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.606288 2583 factory.go:103] Registering Raw factory Sep 10 12:06:18.606373 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.606308 2583 manager.go:1196] Started watching for new ooms in manager Sep 10 12:06:18.606723 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.606700 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lv6lh" Sep 10 12:06:18.606804 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.606742 2583 manager.go:349] Starting recovery of all containers Sep 10 12:06:18.607859 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.607831 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-9.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 12:06:18.608129 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.608099 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 12:06:18.617574 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.617549 2583 manager.go:354] Recovery completed Sep 10 12:06:18.617861 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.617839 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lv6lh" Sep 10 12:06:18.622109 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.622093 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:06:18.623790 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.623758 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:06:18.623883 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.623812 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:06:18.623883 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.623829 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:06:18.624391 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.624378 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 12:06:18.624391 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.624390 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 12:06:18.624450 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.624405 2583 state_mem.go:36] "Initialized new in-memory state store" Sep 10 12:06:18.624450 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.624439 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:06:18.625211 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.625196 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:06:18.625266 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.625227 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:06:18.625266 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.625244 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:06:18.625622 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.625558 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-9.ec2.internal.18d3f43368220f70 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-9.ec2.internal,UID:ip-10-0-141-9.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-9.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-9.ec2.internal,},FirstTimestamp:2026-09-10 12:06:18.62379096 +0000 UTC m=+0.350695500,LastTimestamp:2026-09-10 12:06:18.62379096 +0000 UTC m=+0.350695500,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-9.ec2.internal,}" Sep 10 12:06:18.626432 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.626420 2583 policy_none.go:49] "None policy: Start" Sep 10 12:06:18.626479 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.626436 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 12:06:18.626479 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.626446 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 12:06:18.628429 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.628419 2583 policy_none.go:47] "Start" Sep 10 12:06:18.674407 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.674389 2583 manager.go:337] "Starting Device Plugin manager" Sep 10 12:06:18.692305 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.674441 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 12:06:18.692305 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.674455 2583 server.go:87] "Starting device plugin registration server" Sep 10 12:06:18.692305 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.674737 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 12:06:18.692305 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.674750 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 12:06:18.692305 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.674868 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 12:06:18.692305 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.674967 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 12:06:18.692305 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.674976 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 12:06:18.692305 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.675446 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 12:06:18.692305 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.675499 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:18.744588 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.744548 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 12:06:18.745956 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.745943 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 12:06:18.746063 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.745992 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 12:06:18.746063 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.746022 2583 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 12:06:18.746164 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.746071 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 12:06:18.748823 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.748804 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:06:18.775379 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.775279 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:06:18.776370 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.776352 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:06:18.776472 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.776383 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:06:18.776472 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.776393 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:06:18.776472 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.776414 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.785042 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.785024 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.785129 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.785049 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-9.ec2.internal\": node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:18.809864 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.809836 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:18.846367 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.846333 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal"] Sep 10 12:06:18.846529 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.846414 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:06:18.847372 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.847356 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:06:18.847457 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.847389 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:06:18.847457 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.847403 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:06:18.849890 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.849874 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:06:18.849988 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.849980 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.850035 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.850006 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:06:18.850665 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.850648 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:06:18.850665 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.850658 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:06:18.850796 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.850680 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:06:18.850796 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.850683 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:06:18.850796 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.850690 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:06:18.850796 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.850698 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:06:18.853066 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.853048 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.853116 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.853089 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:06:18.853863 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.853847 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:06:18.853953 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.853877 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:06:18.853953 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.853892 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:06:18.876538 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.876508 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-9.ec2.internal\" not found" node="ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.881440 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.881421 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-9.ec2.internal\" not found" node="ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.905988 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.905947 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e79bb2805f50e77c363fc54fc65fe32-config\") pod \"kube-apiserver-proxy-ip-10-0-141-9.ec2.internal\" (UID: \"4e79bb2805f50e77c363fc54fc65fe32\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.907519 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.906010 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83631e085fe0c12b5412351d1accf045-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal\" (UID: \"83631e085fe0c12b5412351d1accf045\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.907519 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:18.906048 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83631e085fe0c12b5412351d1accf045-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal\" (UID: \"83631e085fe0c12b5412351d1accf045\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" Sep 10 12:06:18.910834 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:18.910807 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:19.006356 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.006317 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e79bb2805f50e77c363fc54fc65fe32-config\") pod \"kube-apiserver-proxy-ip-10-0-141-9.ec2.internal\" (UID: \"4e79bb2805f50e77c363fc54fc65fe32\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.006495 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.006346 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e79bb2805f50e77c363fc54fc65fe32-config\") pod \"kube-apiserver-proxy-ip-10-0-141-9.ec2.internal\" (UID: \"4e79bb2805f50e77c363fc54fc65fe32\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.006495 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.006414 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83631e085fe0c12b5412351d1accf045-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal\" (UID: \"83631e085fe0c12b5412351d1accf045\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.006495 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.006438 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83631e085fe0c12b5412351d1accf045-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal\" (UID: \"83631e085fe0c12b5412351d1accf045\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.006495 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.006490 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83631e085fe0c12b5412351d1accf045-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal\" (UID: \"83631e085fe0c12b5412351d1accf045\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.006638 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.006509 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83631e085fe0c12b5412351d1accf045-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal\" (UID: \"83631e085fe0c12b5412351d1accf045\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.011372 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:19.011358 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:19.112471 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:19.112405 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:19.179689 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.179667 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.184753 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.184568 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.212956 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:19.212925 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:19.313526 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:19.313488 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:19.414169 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:19.414077 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:19.514686 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:19.514654 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:19.541841 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.541814 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 12:06:19.542309 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.542029 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 12:06:19.614944 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:19.614912 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:19.621084 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.621052 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 12:01:19 +0000 UTC" deadline="2028-05-22 22:08:38.54550108 +0000 UTC" Sep 10 12:06:19.621161 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.621083 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14890h2m18.924423171s" Sep 10 12:06:19.650518 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.650493 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:06:19.662093 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.662064 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:06:19.715534 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:19.715463 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-9.ec2.internal\" not found" Sep 10 12:06:19.727035 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.727009 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:06:19.804753 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.804727 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.815462 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.815439 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 12:06:19.816356 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.816342 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" Sep 10 12:06:19.827609 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:19.827583 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 12:06:21.561016 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:21.560839 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83631e085fe0c12b5412351d1accf045.slice/crio-21d8f5fbb18e50aac4d6ce3dd0d84d17c8ed64dbbc01b05aa2713308ff0f39dd WatchSource:0}: Error finding container 21d8f5fbb18e50aac4d6ce3dd0d84d17c8ed64dbbc01b05aa2713308ff0f39dd: Status 404 returned error can't find the container with id 21d8f5fbb18e50aac4d6ce3dd0d84d17c8ed64dbbc01b05aa2713308ff0f39dd Sep 10 12:06:21.565720 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.565706 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:06:21.643636 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:21.643607 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e79bb2805f50e77c363fc54fc65fe32.slice/crio-cb0c97d8223456197f6fe26f2c36840028c6e023b82e63e4533ddb8f1f41114c WatchSource:0}: Error finding container cb0c97d8223456197f6fe26f2c36840028c6e023b82e63e4533ddb8f1f41114c: Status 404 returned error can't find the container with id cb0c97d8223456197f6fe26f2c36840028c6e023b82e63e4533ddb8f1f41114c Sep 10 12:06:21.734311 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.734222 2583 apiserver.go:52] "Watching apiserver" Sep 10 12:06:21.739940 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.739921 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 12:06:21.740260 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.740240 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-s8hwl","openshift-network-operator/iptables-alerter-mcbp8","openshift-ovn-kubernetes/ovnkube-node-2znnx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5","openshift-cluster-node-tuning-operator/tuned-xv2fl","openshift-dns/node-resolver-klmnw","openshift-image-registry/node-ca-bxjqj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal","openshift-network-diagnostics/network-check-target-9wlph","kube-system/global-pull-secret-syncer-7wxnx","kube-system/konnectivity-agent-lg9xf","kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal","openshift-multus/multus-additional-cni-plugins-vsdv2","openshift-multus/multus-fb9x6"] Sep 10 12:06:21.743005 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.742988 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.745013 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.744995 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.747180 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.747155 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 12:06:21.747180 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.747170 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 12:06:21.747323 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.747160 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tx475\"" Sep 10 12:06:21.747323 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.747162 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 12:06:21.747323 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.747253 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.748884 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.748865 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 12:06:21.749040 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.749025 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:06:21.749110 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.749086 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w74g8\"" Sep 10 12:06:21.749166 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.749135 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 12:06:21.749427 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.749405 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:21.749534 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.749476 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:21.749724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.749575 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 12:06:21.749852 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.749836 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 12:06:21.749915 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.749885 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 12:06:21.752467 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.750669 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 12:06:21.752467 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.750722 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 12:06:21.752467 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.751104 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jn5ff\"" Sep 10 12:06:21.753074 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.753052 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 12:06:21.753653 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.753637 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:21.755666 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.755648 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xrx52\"" Sep 10 12:06:21.755892 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.755867 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.756008 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.755991 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 12:06:21.756144 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.756124 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 12:06:21.758141 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.758122 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.759220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.759204 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 12:06:21.759291 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.759247 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 12:06:21.759291 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.759255 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k4l5n\"" Sep 10 12:06:21.759912 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.759897 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 12:06:21.759952 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.759926 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wth2h\"" Sep 10 12:06:21.760228 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.760214 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:21.760298 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.760279 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:21.760515 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.760502 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 12:06:21.760560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.760517 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 12:06:21.762542 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.762528 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:21.762606 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.762584 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:21.763735 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.763716 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 12:06:21.764691 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.764673 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.764927 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.764904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9pz7\" (UniqueName: \"kubernetes.io/projected/31074e24-3342-4c93-94a8-9abc7e0e454e-kube-api-access-l9pz7\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.765014 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.764937 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4psp\" (UniqueName: \"kubernetes.io/projected/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-kube-api-access-s4psp\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:21.765014 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.764955 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/227a6eda-79a6-49f9-9bed-eb32fbeff27a-agent-certs\") pod \"konnectivity-agent-lg9xf\" (UID: \"227a6eda-79a6-49f9-9bed-eb32fbeff27a\") " pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:21.765014 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765007 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/227a6eda-79a6-49f9-9bed-eb32fbeff27a-konnectivity-ca\") pod \"konnectivity-agent-lg9xf\" (UID: \"227a6eda-79a6-49f9-9bed-eb32fbeff27a\") " pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:21.765124 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765030 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-sys-fs\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.765124 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765045 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp2qj\" (UniqueName: \"kubernetes.io/projected/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-kube-api-access-vp2qj\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.765124 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765069 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk6ch\" (UniqueName: \"kubernetes.io/projected/1dfb8b69-72b9-4fb8-b33d-a2b938b653a1-kube-api-access-qk6ch\") pod \"node-ca-bxjqj\" (UID: \"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1\") " pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.765226 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765130 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2605e86c-70b6-4188-8c87-4f8629dafbd7-host-slash\") pod \"iptables-alerter-mcbp8\" (UID: \"2605e86c-70b6-4188-8c87-4f8629dafbd7\") " pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.765226 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/31074e24-3342-4c93-94a8-9abc7e0e454e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.765226 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31074e24-3342-4c93-94a8-9abc7e0e454e-cnibin\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.765226 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765209 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31074e24-3342-4c93-94a8-9abc7e0e454e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.765390 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765230 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:21.765390 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765270 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.765390 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765284 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-registration-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.765390 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765304 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-device-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.765390 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765345 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dfb8b69-72b9-4fb8-b33d-a2b938b653a1-host\") pod \"node-ca-bxjqj\" (UID: \"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1\") " pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.765560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765391 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2605e86c-70b6-4188-8c87-4f8629dafbd7-iptables-alerter-script\") pod \"iptables-alerter-mcbp8\" (UID: \"2605e86c-70b6-4188-8c87-4f8629dafbd7\") " pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.765560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765409 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwl4s\" (UniqueName: \"kubernetes.io/projected/2605e86c-70b6-4188-8c87-4f8629dafbd7-kube-api-access-zwl4s\") pod \"iptables-alerter-mcbp8\" (UID: \"2605e86c-70b6-4188-8c87-4f8629dafbd7\") " pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.765560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765423 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31074e24-3342-4c93-94a8-9abc7e0e454e-system-cni-dir\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.765560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765448 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31074e24-3342-4c93-94a8-9abc7e0e454e-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.765560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765465 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-socket-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.765560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765480 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-etc-selinux\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.765560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765494 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1dfb8b69-72b9-4fb8-b33d-a2b938b653a1-serviceca\") pod \"node-ca-bxjqj\" (UID: \"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1\") " pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.765560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.765512 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/31074e24-3342-4c93-94a8-9abc7e0e454e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.766846 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.766830 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 12:06:21.766998 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.766981 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:06:21.767089 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.767074 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vm2p4\"" Sep 10 12:06:21.767147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.767130 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.769268 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.769252 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.769617 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.769587 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 12:06:21.769707 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.769629 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 12:06:21.769707 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.769693 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cp2sj\"" Sep 10 12:06:21.769813 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.769795 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 12:06:21.769813 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.769800 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 12:06:21.769922 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.769812 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 12:06:21.769977 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.769941 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 12:06:21.771416 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.771401 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 12:06:21.771536 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.771520 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hp4tw\"" Sep 10 12:06:21.801840 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.801820 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t97m5" Sep 10 12:06:21.813134 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.813109 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t97m5" Sep 10 12:06:21.855475 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.855443 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 12:06:21.865719 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865682 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-node-log\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.865719 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865718 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.865914 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865748 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31074e24-3342-4c93-94a8-9abc7e0e454e-cnibin\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.865914 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865805 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31074e24-3342-4c93-94a8-9abc7e0e454e-cnibin\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.865914 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865804 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31074e24-3342-4c93-94a8-9abc7e0e454e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.865914 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865844 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-system-cni-dir\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.865914 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865865 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/216eee13-735c-4c33-8377-40dacf2c051d-etc-tuned\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.865914 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865884 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-systemd-units\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.865914 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865902 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-run-netns\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865924 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-cni-netd\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865945 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.865970 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-registration-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866018 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866028 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-run\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866064 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-run-openvswitch\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866068 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-registration-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866091 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-cni-dir\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866112 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-kubelet-config\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866136 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-modprobe-d\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.866157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866158 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-sysctl-d\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-var-lib-kubelet\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866191 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-run-ovn-kubernetes\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866206 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-env-overrides\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866231 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/216eee13-735c-4c33-8377-40dacf2c051d-tmp\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866251 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-kubelet\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866270 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vk7n\" (UniqueName: \"kubernetes.io/projected/dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3-kube-api-access-2vk7n\") pod \"node-resolver-klmnw\" (UID: \"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3\") " pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31074e24-3342-4c93-94a8-9abc7e0e454e-system-cni-dir\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866320 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/31074e24-3342-4c93-94a8-9abc7e0e454e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866344 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-cnibin\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866380 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31074e24-3342-4c93-94a8-9abc7e0e454e-system-cni-dir\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866386 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-var-lib-kubelet\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866420 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-kubernetes\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866495 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpbpf\" (UniqueName: \"kubernetes.io/projected/216eee13-735c-4c33-8377-40dacf2c051d-kube-api-access-hpbpf\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866534 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-slash\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.866596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866584 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-run-systemd\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866612 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-etc-selinux\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866640 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/227a6eda-79a6-49f9-9bed-eb32fbeff27a-agent-certs\") pod \"konnectivity-agent-lg9xf\" (UID: \"227a6eda-79a6-49f9-9bed-eb32fbeff27a\") " pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866663 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-run-multus-certs\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866642 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31074e24-3342-4c93-94a8-9abc7e0e454e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866716 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-var-lib-openvswitch\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866732 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-etc-selinux\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866812 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-run-ovn\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866848 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-ovnkube-config\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866872 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-sysconfig\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/31074e24-3342-4c93-94a8-9abc7e0e454e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866925 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/31074e24-3342-4c93-94a8-9abc7e0e454e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866952 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-cni-binary-copy\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866981 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-var-lib-cni-bin\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.866999 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3-tmp-dir\") pod \"node-resolver-klmnw\" (UID: \"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3\") " pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867017 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-socket-dir-parent\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.867150 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867076 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-run-netns\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867091 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-device-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867112 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2605e86c-70b6-4188-8c87-4f8629dafbd7-iptables-alerter-script\") pod \"iptables-alerter-mcbp8\" (UID: \"2605e86c-70b6-4188-8c87-4f8629dafbd7\") " pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867184 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwl4s\" (UniqueName: \"kubernetes.io/projected/2605e86c-70b6-4188-8c87-4f8629dafbd7-kube-api-access-zwl4s\") pod \"iptables-alerter-mcbp8\" (UID: \"2605e86c-70b6-4188-8c87-4f8629dafbd7\") " pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867192 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-device-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867225 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9pz7\" (UniqueName: \"kubernetes.io/projected/31074e24-3342-4c93-94a8-9abc7e0e454e-kube-api-access-l9pz7\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867251 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/227a6eda-79a6-49f9-9bed-eb32fbeff27a-konnectivity-ca\") pod \"konnectivity-agent-lg9xf\" (UID: \"227a6eda-79a6-49f9-9bed-eb32fbeff27a\") " pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.867277 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867279 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-etc-kubernetes\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867340 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/31074e24-3342-4c93-94a8-9abc7e0e454e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.867381 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs podName:35e290b1-c39f-4ab8-b6ed-9584e49b3ae4 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:22.367339233 +0000 UTC m=+2.944662839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs") pod "network-metrics-daemon-s8hwl" (UID: "35e290b1-c39f-4ab8-b6ed-9584e49b3ae4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867412 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-dbus\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867411 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867445 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-lib-modules\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867513 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4psp\" (UniqueName: \"kubernetes.io/projected/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-kube-api-access-s4psp\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867536 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2dwp\" (UniqueName: \"kubernetes.io/projected/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-kube-api-access-g2dwp\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867561 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-sysctl-conf\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.867849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867582 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-sys\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867602 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-log-socket\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867620 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-cni-bin\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867639 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-ovnkube-script-lib\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867651 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2605e86c-70b6-4188-8c87-4f8629dafbd7-iptables-alerter-script\") pod \"iptables-alerter-mcbp8\" (UID: \"2605e86c-70b6-4188-8c87-4f8629dafbd7\") " pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-sys-fs\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867694 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-daemon-config\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867722 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-host\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867730 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-sys-fs\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867749 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpz4l\" (UniqueName: \"kubernetes.io/projected/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-kube-api-access-tpz4l\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867772 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-run-k8s-cni-cncf-io\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867786 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867801 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-ovn-node-metrics-cert\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867834 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dfb8b69-72b9-4fb8-b33d-a2b938b653a1-host\") pod \"node-ca-bxjqj\" (UID: \"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1\") " pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867851 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31074e24-3342-4c93-94a8-9abc7e0e454e-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867866 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-hostroot\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-systemd\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.868486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867906 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1dfb8b69-72b9-4fb8-b33d-a2b938b653a1-host\") pod \"node-ca-bxjqj\" (UID: \"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1\") " pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867937 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/227a6eda-79a6-49f9-9bed-eb32fbeff27a-konnectivity-ca\") pod \"konnectivity-agent-lg9xf\" (UID: \"227a6eda-79a6-49f9-9bed-eb32fbeff27a\") " pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.867987 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868016 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-etc-openvswitch\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868040 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-socket-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868067 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1dfb8b69-72b9-4fb8-b33d-a2b938b653a1-serviceca\") pod \"node-ca-bxjqj\" (UID: \"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1\") " pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868091 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-var-lib-cni-multus\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868116 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3-hosts-file\") pod \"node-resolver-klmnw\" (UID: \"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3\") " pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868151 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-socket-dir\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868168 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp2qj\" (UniqueName: \"kubernetes.io/projected/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-kube-api-access-vp2qj\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868200 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk6ch\" (UniqueName: \"kubernetes.io/projected/1dfb8b69-72b9-4fb8-b33d-a2b938b653a1-kube-api-access-qk6ch\") pod \"node-ca-bxjqj\" (UID: \"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1\") " pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868225 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2605e86c-70b6-4188-8c87-4f8629dafbd7-host-slash\") pod \"iptables-alerter-mcbp8\" (UID: \"2605e86c-70b6-4188-8c87-4f8629dafbd7\") " pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868251 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-conf-dir\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868274 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31074e24-3342-4c93-94a8-9abc7e0e454e-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868318 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2605e86c-70b6-4188-8c87-4f8629dafbd7-host-slash\") pod \"iptables-alerter-mcbp8\" (UID: \"2605e86c-70b6-4188-8c87-4f8629dafbd7\") " pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.868962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.868480 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1dfb8b69-72b9-4fb8-b33d-a2b938b653a1-serviceca\") pod \"node-ca-bxjqj\" (UID: \"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1\") " pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.871147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.871130 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/227a6eda-79a6-49f9-9bed-eb32fbeff27a-agent-certs\") pod \"konnectivity-agent-lg9xf\" (UID: \"227a6eda-79a6-49f9-9bed-eb32fbeff27a\") " pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:21.878223 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.878201 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwl4s\" (UniqueName: \"kubernetes.io/projected/2605e86c-70b6-4188-8c87-4f8629dafbd7-kube-api-access-zwl4s\") pod \"iptables-alerter-mcbp8\" (UID: \"2605e86c-70b6-4188-8c87-4f8629dafbd7\") " pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:21.878335 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.878263 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9pz7\" (UniqueName: \"kubernetes.io/projected/31074e24-3342-4c93-94a8-9abc7e0e454e-kube-api-access-l9pz7\") pod \"multus-additional-cni-plugins-vsdv2\" (UID: \"31074e24-3342-4c93-94a8-9abc7e0e454e\") " pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:21.878833 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.878818 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp2qj\" (UniqueName: \"kubernetes.io/projected/7b836b1e-a88e-4d6b-8c71-6a0fc025b52b-kube-api-access-vp2qj\") pod \"aws-ebs-csi-driver-node-8tqc5\" (UID: \"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:21.889643 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.889615 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk6ch\" (UniqueName: \"kubernetes.io/projected/1dfb8b69-72b9-4fb8-b33d-a2b938b653a1-kube-api-access-qk6ch\") pod \"node-ca-bxjqj\" (UID: \"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1\") " pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:21.889754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.889718 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4psp\" (UniqueName: \"kubernetes.io/projected/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-kube-api-access-s4psp\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:21.900813 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.900769 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal" event={"ID":"4e79bb2805f50e77c363fc54fc65fe32","Type":"ContainerStarted","Data":"cb0c97d8223456197f6fe26f2c36840028c6e023b82e63e4533ddb8f1f41114c"} Sep 10 12:06:21.901832 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.901812 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" event={"ID":"83631e085fe0c12b5412351d1accf045","Type":"ContainerStarted","Data":"21d8f5fbb18e50aac4d6ce3dd0d84d17c8ed64dbbc01b05aa2713308ff0f39dd"} Sep 10 12:06:21.968897 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.968870 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-conf-dir\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.969025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.968901 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-node-log\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.968921 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.968937 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-system-cni-dir\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.969025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.968956 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/216eee13-735c-4c33-8377-40dacf2c051d-etc-tuned\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.969025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.968980 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-systemd-units\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.968992 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-conf-dir\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.969025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969006 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-run-netns\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.968993 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-node-log\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969031 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-cni-netd\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969006 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969033 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-systemd-units\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969064 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-cni-netd\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969069 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-run-netns\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969067 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-run\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969111 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-system-cni-dir\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-run-openvswitch\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969148 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-cni-dir\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969156 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-run\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969165 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-run-openvswitch\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969176 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-kubelet-config\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969207 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-modprobe-d\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969224 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-sysctl-d\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969235 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-kubelet-config\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969240 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-var-lib-kubelet\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969278 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-run-ovn-kubernetes\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.969420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969286 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-var-lib-kubelet\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969306 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-env-overrides\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969310 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-modprobe-d\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969322 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-run-ovn-kubernetes\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969324 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-cni-dir\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969330 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/216eee13-735c-4c33-8377-40dacf2c051d-tmp\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969393 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-kubelet\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969396 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-sysctl-d\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vk7n\" (UniqueName: \"kubernetes.io/projected/dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3-kube-api-access-2vk7n\") pod \"node-resolver-klmnw\" (UID: \"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3\") " pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969449 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-kubelet\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969451 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-cnibin\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969494 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-var-lib-kubelet\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969518 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-kubernetes\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969531 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-var-lib-kubelet\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969495 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-cnibin\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969543 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpbpf\" (UniqueName: \"kubernetes.io/projected/216eee13-735c-4c33-8377-40dacf2c051d-kube-api-access-hpbpf\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969614 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-kubernetes\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969658 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-slash\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.970263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969681 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-run-systemd\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-run-multus-certs\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969764 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-run-multus-certs\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969808 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-env-overrides\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969846 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-slash\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969893 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-var-lib-openvswitch\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969911 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-var-lib-openvswitch\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969920 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-run-ovn\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969935 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-run-systemd\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969947 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-ovnkube-config\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969971 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-sysconfig\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969978 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-run-ovn\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.969999 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-cni-binary-copy\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970024 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-var-lib-cni-bin\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970049 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3-tmp-dir\") pod \"node-resolver-klmnw\" (UID: \"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3\") " pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970096 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-socket-dir-parent\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970119 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-var-lib-cni-bin\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970124 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-run-netns\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970153 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-etc-kubernetes\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970175 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-dbus\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970202 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-lib-modules\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970242 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2dwp\" (UniqueName: \"kubernetes.io/projected/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-kube-api-access-g2dwp\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-sysctl-conf\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970287 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-sys\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970308 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-log-socket\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970315 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3-tmp-dir\") pod \"node-resolver-klmnw\" (UID: \"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3\") " pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970330 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-cni-bin\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970383 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-ovnkube-script-lib\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970412 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-daemon-config\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970442 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-lib-modules\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970436 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-host\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970497 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpz4l\" (UniqueName: \"kubernetes.io/projected/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-kube-api-access-tpz4l\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970512 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-dbus\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970521 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-run-k8s-cni-cncf-io\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970546 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970550 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-socket-dir-parent\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.971967 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970570 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-ovn-node-metrics-cert\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970594 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-hostroot\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970623 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-cni-binary-copy\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-systemd\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970672 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970695 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-etc-kubernetes\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970711 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-etc-openvswitch\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970735 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-var-lib-cni-multus\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3-hosts-file\") pod \"node-resolver-klmnw\" (UID: \"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3\") " pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970787 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-sys\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970897 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-log-socket\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970684 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-ovnkube-config\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971255 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-ovnkube-script-lib\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.970730 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-run-netns\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971422 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-var-lib-cni-multus\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971428 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-etc-openvswitch\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971427 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-sysconfig\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971460 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-host-cni-bin\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.972629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971475 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3-hosts-file\") pod \"node-resolver-klmnw\" (UID: \"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3\") " pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971527 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-host-run-k8s-cni-cncf-io\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971572 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-multus-daemon-config\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971655 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-systemd\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971711 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-host\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.971741 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971750 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/216eee13-735c-4c33-8377-40dacf2c051d-etc-sysctl-conf\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.971798 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret podName:faf4ab54-b2cc-42bf-a906-e23f8d7949d9 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:22.471782916 +0000 UTC m=+3.049106505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret") pod "global-pull-secret-syncer-7wxnx" (UID: "faf4ab54-b2cc-42bf-a906-e23f8d7949d9") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/216eee13-735c-4c33-8377-40dacf2c051d-etc-tuned\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971878 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/216eee13-735c-4c33-8377-40dacf2c051d-tmp\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.973114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.971901 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-hostroot\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:21.973716 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.973700 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-ovn-node-metrics-cert\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.988688 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.988636 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:06:21.988688 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.988659 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:06:21.988688 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.988669 2583 projected.go:196] Error preparing data for projected volume kube-api-access-gb2ck for pod openshift-network-diagnostics/network-check-target-9wlph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:21.988866 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:21.988719 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck podName:3323f642-5184-411e-8db6-d639e9259107 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:22.488706224 +0000 UTC m=+3.066029811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gb2ck" (UniqueName: "kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck") pod "network-check-target-9wlph" (UID: "3323f642-5184-411e-8db6-d639e9259107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:21.988943 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.988926 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpz4l\" (UniqueName: \"kubernetes.io/projected/924a1e16-0dea-4f1d-9ec2-2dc240faff7d-kube-api-access-tpz4l\") pod \"ovnkube-node-2znnx\" (UID: \"924a1e16-0dea-4f1d-9ec2-2dc240faff7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:21.989932 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.989912 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpbpf\" (UniqueName: \"kubernetes.io/projected/216eee13-735c-4c33-8377-40dacf2c051d-kube-api-access-hpbpf\") pod \"tuned-xv2fl\" (UID: \"216eee13-735c-4c33-8377-40dacf2c051d\") " pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:21.991535 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.991517 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vk7n\" (UniqueName: \"kubernetes.io/projected/dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3-kube-api-access-2vk7n\") pod \"node-resolver-klmnw\" (UID: \"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3\") " pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:21.991662 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:21.991643 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2dwp\" (UniqueName: \"kubernetes.io/projected/0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108-kube-api-access-g2dwp\") pod \"multus-fb9x6\" (UID: \"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108\") " pod="openshift-multus/multus-fb9x6" Sep 10 12:06:22.054832 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.054792 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bxjqj" Sep 10 12:06:22.061394 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.061374 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mcbp8" Sep 10 12:06:22.066183 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:22.066162 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dfb8b69_72b9_4fb8_b33d_a2b938b653a1.slice/crio-26394caf395c40c4e732c9631fedfa2b98ad7c9d5bf12feb9f0dc27474d30d66 WatchSource:0}: Error finding container 26394caf395c40c4e732c9631fedfa2b98ad7c9d5bf12feb9f0dc27474d30d66: Status 404 returned error can't find the container with id 26394caf395c40c4e732c9631fedfa2b98ad7c9d5bf12feb9f0dc27474d30d66 Sep 10 12:06:22.068448 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.068331 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" Sep 10 12:06:22.070513 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.070493 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:06:22.070893 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:22.070816 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2605e86c_70b6_4188_8c87_4f8629dafbd7.slice/crio-df39e87f2468e3652eda33560744416ee6e59c203be0f91fa9947bee70c8fc14 WatchSource:0}: Error finding container df39e87f2468e3652eda33560744416ee6e59c203be0f91fa9947bee70c8fc14: Status 404 returned error can't find the container with id df39e87f2468e3652eda33560744416ee6e59c203be0f91fa9947bee70c8fc14 Sep 10 12:06:22.073724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.073704 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:22.077910 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:22.077888 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31074e24_3342_4c93_94a8_9abc7e0e454e.slice/crio-a223587c727caaaf2b692ee019cb45c067b505b1d253160eb145a636d784d171 WatchSource:0}: Error finding container a223587c727caaaf2b692ee019cb45c067b505b1d253160eb145a636d784d171: Status 404 returned error can't find the container with id a223587c727caaaf2b692ee019cb45c067b505b1d253160eb145a636d784d171 Sep 10 12:06:22.080043 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.080024 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" Sep 10 12:06:22.082967 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:22.082897 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod227a6eda_79a6_49f9_9bed_eb32fbeff27a.slice/crio-f940f888b9f9e6b25293793c2929d54f389228502c5ce9b166974035b790c368 WatchSource:0}: Error finding container f940f888b9f9e6b25293793c2929d54f389228502c5ce9b166974035b790c368: Status 404 returned error can't find the container with id f940f888b9f9e6b25293793c2929d54f389228502c5ce9b166974035b790c368 Sep 10 12:06:22.085120 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.085072 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-klmnw" Sep 10 12:06:22.089908 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:22.089887 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b836b1e_a88e_4d6b_8c71_6a0fc025b52b.slice/crio-2cf5f20471293f8c3ed1ed6619c8d83a190027ea4688eabe51d27316cc05e173 WatchSource:0}: Error finding container 2cf5f20471293f8c3ed1ed6619c8d83a190027ea4688eabe51d27316cc05e173: Status 404 returned error can't find the container with id 2cf5f20471293f8c3ed1ed6619c8d83a190027ea4688eabe51d27316cc05e173 Sep 10 12:06:22.090792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.090776 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" Sep 10 12:06:22.095101 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:22.095079 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcea16cd_6eca_4e99_8c4e_f5d1f7d956e3.slice/crio-a4412cedfdbdc1e2830044ac15ff4ac936f87281b0e22db08a539f0434fb499e WatchSource:0}: Error finding container a4412cedfdbdc1e2830044ac15ff4ac936f87281b0e22db08a539f0434fb499e: Status 404 returned error can't find the container with id a4412cedfdbdc1e2830044ac15ff4ac936f87281b0e22db08a539f0434fb499e Sep 10 12:06:22.095733 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.095710 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:22.100073 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.100036 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fb9x6" Sep 10 12:06:22.101542 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:22.101519 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216eee13_735c_4c33_8377_40dacf2c051d.slice/crio-98009d29e5accb53269dafc1e18b937f112673d9cf4f8b25ec02563e0284f9ee WatchSource:0}: Error finding container 98009d29e5accb53269dafc1e18b937f112673d9cf4f8b25ec02563e0284f9ee: Status 404 returned error can't find the container with id 98009d29e5accb53269dafc1e18b937f112673d9cf4f8b25ec02563e0284f9ee Sep 10 12:06:22.107140 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:22.107111 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924a1e16_0dea_4f1d_9ec2_2dc240faff7d.slice/crio-076e8b16e7519e625175cb9fbd68968372d77aa06046f4007da92d78ef6cffaf WatchSource:0}: Error finding container 076e8b16e7519e625175cb9fbd68968372d77aa06046f4007da92d78ef6cffaf: Status 404 returned error can't find the container with id 076e8b16e7519e625175cb9fbd68968372d77aa06046f4007da92d78ef6cffaf Sep 10 12:06:22.109330 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:06:22.109264 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb91a8b_4cf7_4e08_b9e3_f5d46b6b5108.slice/crio-c8b9e298eb5893d1ed25b15c1d48a305eba54f474ec363c015353b1076c94bbe WatchSource:0}: Error finding container c8b9e298eb5893d1ed25b15c1d48a305eba54f474ec363c015353b1076c94bbe: Status 404 returned error can't find the container with id c8b9e298eb5893d1ed25b15c1d48a305eba54f474ec363c015353b1076c94bbe Sep 10 12:06:22.373548 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.373457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:22.373747 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:22.373589 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:22.373747 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:22.373652 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs podName:35e290b1-c39f-4ab8-b6ed-9584e49b3ae4 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:23.37363577 +0000 UTC m=+3.950959379 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs") pod "network-metrics-daemon-s8hwl" (UID: "35e290b1-c39f-4ab8-b6ed-9584e49b3ae4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:22.474157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.474111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:22.474335 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:22.474264 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:22.474335 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:22.474316 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret podName:faf4ab54-b2cc-42bf-a906-e23f8d7949d9 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:23.474304023 +0000 UTC m=+4.051627624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret") pod "global-pull-secret-syncer-7wxnx" (UID: "faf4ab54-b2cc-42bf-a906-e23f8d7949d9") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:22.579906 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.579861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:22.581367 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:22.580085 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:06:22.581367 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:22.580116 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:06:22.581367 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:22.580131 2583 projected.go:196] Error preparing data for projected volume kube-api-access-gb2ck for pod openshift-network-diagnostics/network-check-target-9wlph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:22.581367 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:22.580200 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck podName:3323f642-5184-411e-8db6-d639e9259107 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:23.580181116 +0000 UTC m=+4.157504711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gb2ck" (UniqueName: "kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck") pod "network-check-target-9wlph" (UID: "3323f642-5184-411e-8db6-d639e9259107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:22.818202 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.818082 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:01:21 +0000 UTC" deadline="2028-04-01 19:35:28.865833736 +0000 UTC" Sep 10 12:06:22.818202 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.818128 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13663h29m6.047710157s" Sep 10 12:06:22.896792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.896395 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:22.896792 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:22.896549 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:22.906607 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.906567 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bxjqj" event={"ID":"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1","Type":"ContainerStarted","Data":"26394caf395c40c4e732c9631fedfa2b98ad7c9d5bf12feb9f0dc27474d30d66"} Sep 10 12:06:22.910681 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.910648 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" event={"ID":"924a1e16-0dea-4f1d-9ec2-2dc240faff7d","Type":"ContainerStarted","Data":"076e8b16e7519e625175cb9fbd68968372d77aa06046f4007da92d78ef6cffaf"} Sep 10 12:06:22.925439 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.925395 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" event={"ID":"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b","Type":"ContainerStarted","Data":"2cf5f20471293f8c3ed1ed6619c8d83a190027ea4688eabe51d27316cc05e173"} Sep 10 12:06:22.927399 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.927329 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lg9xf" event={"ID":"227a6eda-79a6-49f9-9bed-eb32fbeff27a","Type":"ContainerStarted","Data":"f940f888b9f9e6b25293793c2929d54f389228502c5ce9b166974035b790c368"} Sep 10 12:06:22.929478 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.929450 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fb9x6" event={"ID":"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108","Type":"ContainerStarted","Data":"c8b9e298eb5893d1ed25b15c1d48a305eba54f474ec363c015353b1076c94bbe"} Sep 10 12:06:22.933265 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.933197 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" event={"ID":"216eee13-735c-4c33-8377-40dacf2c051d","Type":"ContainerStarted","Data":"98009d29e5accb53269dafc1e18b937f112673d9cf4f8b25ec02563e0284f9ee"} Sep 10 12:06:22.939747 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.939675 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-klmnw" event={"ID":"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3","Type":"ContainerStarted","Data":"a4412cedfdbdc1e2830044ac15ff4ac936f87281b0e22db08a539f0434fb499e"} Sep 10 12:06:22.943371 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.943286 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" event={"ID":"31074e24-3342-4c93-94a8-9abc7e0e454e","Type":"ContainerStarted","Data":"a223587c727caaaf2b692ee019cb45c067b505b1d253160eb145a636d784d171"} Sep 10 12:06:22.945172 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:22.945146 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mcbp8" event={"ID":"2605e86c-70b6-4188-8c87-4f8629dafbd7","Type":"ContainerStarted","Data":"df39e87f2468e3652eda33560744416ee6e59c203be0f91fa9947bee70c8fc14"} Sep 10 12:06:23.386855 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:23.386793 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:23.387058 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.386999 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:23.387142 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.387074 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs podName:35e290b1-c39f-4ab8-b6ed-9584e49b3ae4 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:25.387049764 +0000 UTC m=+5.964373355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs") pod "network-metrics-daemon-s8hwl" (UID: "35e290b1-c39f-4ab8-b6ed-9584e49b3ae4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:23.487727 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:23.487694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:23.487919 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.487871 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:23.487976 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.487932 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret podName:faf4ab54-b2cc-42bf-a906-e23f8d7949d9 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:25.487907591 +0000 UTC m=+6.065231194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret") pod "global-pull-secret-syncer-7wxnx" (UID: "faf4ab54-b2cc-42bf-a906-e23f8d7949d9") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:23.588769 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:23.588731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:23.588948 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.588912 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:06:23.588948 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.588942 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:06:23.589061 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.588957 2583 projected.go:196] Error preparing data for projected volume kube-api-access-gb2ck for pod openshift-network-diagnostics/network-check-target-9wlph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:23.589061 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.589024 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck podName:3323f642-5184-411e-8db6-d639e9259107 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:25.589003466 +0000 UTC m=+6.166327067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gb2ck" (UniqueName: "kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck") pod "network-check-target-9wlph" (UID: "3323f642-5184-411e-8db6-d639e9259107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:23.818886 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:23.818776 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:01:21 +0000 UTC" deadline="2028-03-26 16:53:18.083340857 +0000 UTC" Sep 10 12:06:23.818886 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:23.818821 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13516h46m54.264525341s" Sep 10 12:06:23.898283 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:23.898229 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:23.898476 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.898384 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:23.898770 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:23.898729 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:23.898856 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:23.898798 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:24.896554 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:24.896061 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:24.896554 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:24.896193 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:25.404873 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:25.404835 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:25.405130 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.405097 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:25.405215 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.405169 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs podName:35e290b1-c39f-4ab8-b6ed-9584e49b3ae4 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:29.405149416 +0000 UTC m=+9.982473027 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs") pod "network-metrics-daemon-s8hwl" (UID: "35e290b1-c39f-4ab8-b6ed-9584e49b3ae4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:25.506490 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:25.505927 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:25.506490 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.506073 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:25.506490 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.506137 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret podName:faf4ab54-b2cc-42bf-a906-e23f8d7949d9 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:29.506118631 +0000 UTC m=+10.083442224 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret") pod "global-pull-secret-syncer-7wxnx" (UID: "faf4ab54-b2cc-42bf-a906-e23f8d7949d9") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:25.607384 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:25.607326 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:25.607573 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.607528 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:06:25.607573 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.607551 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:06:25.607573 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.607565 2583 projected.go:196] Error preparing data for projected volume kube-api-access-gb2ck for pod openshift-network-diagnostics/network-check-target-9wlph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:25.607773 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.607629 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck podName:3323f642-5184-411e-8db6-d639e9259107 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:29.607609151 +0000 UTC m=+10.184932742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gb2ck" (UniqueName: "kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck") pod "network-check-target-9wlph" (UID: "3323f642-5184-411e-8db6-d639e9259107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:25.896194 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:25.896085 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:25.896194 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:25.896098 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:25.896433 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.896212 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:25.896433 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:25.896291 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:26.896335 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:26.896298 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:26.896787 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:26.896466 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:27.896109 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:27.896077 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:27.896285 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:27.896121 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:27.896285 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:27.896212 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:27.896808 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:27.896335 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:28.896253 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:28.896223 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:28.897083 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:28.896477 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:29.439436 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:29.439378 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:29.439636 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.439548 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:29.439636 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.439620 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs podName:35e290b1-c39f-4ab8-b6ed-9584e49b3ae4 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:37.439604696 +0000 UTC m=+18.016928288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs") pod "network-metrics-daemon-s8hwl" (UID: "35e290b1-c39f-4ab8-b6ed-9584e49b3ae4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:29.540183 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:29.540140 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:29.540392 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.540345 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:29.540474 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.540428 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret podName:faf4ab54-b2cc-42bf-a906-e23f8d7949d9 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:37.540410357 +0000 UTC m=+18.117733950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret") pod "global-pull-secret-syncer-7wxnx" (UID: "faf4ab54-b2cc-42bf-a906-e23f8d7949d9") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:29.641364 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:29.641305 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:29.641542 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.641525 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:06:29.641599 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.641549 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:06:29.641599 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.641561 2583 projected.go:196] Error preparing data for projected volume kube-api-access-gb2ck for pod openshift-network-diagnostics/network-check-target-9wlph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:29.641685 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.641618 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck podName:3323f642-5184-411e-8db6-d639e9259107 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:37.641601597 +0000 UTC m=+18.218925190 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gb2ck" (UniqueName: "kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck") pod "network-check-target-9wlph" (UID: "3323f642-5184-411e-8db6-d639e9259107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:29.899875 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:29.899224 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:29.899875 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.899334 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:29.899875 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:29.899730 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:29.899875 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:29.899818 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:30.896476 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:30.896436 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:30.896655 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:30.896593 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:31.896646 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:31.896605 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:31.897124 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:31.896611 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:31.897124 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:31.896737 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:31.897124 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:31.896790 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:32.896751 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:32.896715 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:32.897170 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:32.896851 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:33.896204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:33.896165 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:33.896434 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:33.896305 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:33.896434 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:33.896365 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:33.896527 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:33.896440 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:34.895982 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:34.895944 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:34.896476 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:34.896080 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:35.897491 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:35.897457 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:35.897938 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:35.897470 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:35.897938 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:35.897600 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:35.897938 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:35.897680 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:36.895947 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:36.895913 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:36.896130 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:36.896022 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:37.503008 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:37.502967 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:37.503493 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.503144 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:37.503493 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.503227 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs podName:35e290b1-c39f-4ab8-b6ed-9584e49b3ae4 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:53.503205518 +0000 UTC m=+34.080529112 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs") pod "network-metrics-daemon-s8hwl" (UID: "35e290b1-c39f-4ab8-b6ed-9584e49b3ae4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:37.603688 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:37.603644 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:37.603922 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.603796 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:37.603922 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.603868 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret podName:faf4ab54-b2cc-42bf-a906-e23f8d7949d9 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:53.60385275 +0000 UTC m=+34.181176337 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret") pod "global-pull-secret-syncer-7wxnx" (UID: "faf4ab54-b2cc-42bf-a906-e23f8d7949d9") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:37.704850 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:37.704811 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:37.705016 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.704990 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:06:37.705071 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.705022 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:06:37.705071 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.705036 2583 projected.go:196] Error preparing data for projected volume kube-api-access-gb2ck for pod openshift-network-diagnostics/network-check-target-9wlph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:37.705151 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.705092 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck podName:3323f642-5184-411e-8db6-d639e9259107 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:53.705078508 +0000 UTC m=+34.282402094 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gb2ck" (UniqueName: "kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck") pod "network-check-target-9wlph" (UID: "3323f642-5184-411e-8db6-d639e9259107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:37.896210 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:37.896170 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:37.896210 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:37.896222 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:37.896487 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.896337 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:37.896487 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:37.896462 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:38.896139 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:38.896099 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:38.896539 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:38.896214 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:39.896436 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:39.896395 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:39.897014 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:39.896399 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:39.897014 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:39.896508 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:39.897014 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:39.896635 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:40.895973 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:40.895939 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:40.896140 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:40.896076 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:41.897065 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.896752 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:41.897749 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.896813 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:41.897749 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:41.897172 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:41.897749 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:41.897203 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:41.984636 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.984601 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-klmnw" event={"ID":"dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3","Type":"ContainerStarted","Data":"8a88fd34a70c801f1c46e17cda830d659d9d2fce35ddbf131c5f350bd3e6e264"} Sep 10 12:06:41.986289 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.986265 2583 generic.go:356] "Generic (PLEG): container finished" podID="31074e24-3342-4c93-94a8-9abc7e0e454e" containerID="c555fcf0108fc7b1938435a536a30a4fe4fba7731cc15cee44540c64b1a6640e" exitCode=0 Sep 10 12:06:41.986462 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.986335 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" event={"ID":"31074e24-3342-4c93-94a8-9abc7e0e454e","Type":"ContainerDied","Data":"c555fcf0108fc7b1938435a536a30a4fe4fba7731cc15cee44540c64b1a6640e"} Sep 10 12:06:41.988324 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.987936 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bxjqj" event={"ID":"1dfb8b69-72b9-4fb8-b33d-a2b938b653a1","Type":"ContainerStarted","Data":"816809570f595a93c14830392e2d307323fefc0aa70f97d415b279da81592fe1"} Sep 10 12:06:41.989749 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.989728 2583 generic.go:356] "Generic (PLEG): container finished" podID="83631e085fe0c12b5412351d1accf045" containerID="b3383a63fba90cebdbbbb09e6fd726057c90433e5f38364de22623911d6af4a4" exitCode=0 Sep 10 12:06:41.989842 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.989795 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" event={"ID":"83631e085fe0c12b5412351d1accf045","Type":"ContainerDied","Data":"b3383a63fba90cebdbbbb09e6fd726057c90433e5f38364de22623911d6af4a4"} Sep 10 12:06:41.994848 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.993098 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" event={"ID":"924a1e16-0dea-4f1d-9ec2-2dc240faff7d","Type":"ContainerStarted","Data":"5e66ca1e985c969bfb523c193b42f6f0790c27e4eb1d831790ad794cf444c5fa"} Sep 10 12:06:41.994848 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.993173 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" event={"ID":"924a1e16-0dea-4f1d-9ec2-2dc240faff7d","Type":"ContainerStarted","Data":"9e33648a6f8c842570e45b4e516e5175a17e5d73d99217c55212e90ca6ee1394"} Sep 10 12:06:41.994848 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.993223 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" event={"ID":"924a1e16-0dea-4f1d-9ec2-2dc240faff7d","Type":"ContainerStarted","Data":"38fa5d080e26fa509bed70dfbbc19747a8c82f664ab8ce903ab8aa7fff8f7f5e"} Sep 10 12:06:41.994848 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.993270 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" event={"ID":"924a1e16-0dea-4f1d-9ec2-2dc240faff7d","Type":"ContainerStarted","Data":"7d4dd560d8708574ed7968f127dcd6fa20c4481e501f3489700997f7d7066751"} Sep 10 12:06:41.994848 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.993317 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" event={"ID":"924a1e16-0dea-4f1d-9ec2-2dc240faff7d","Type":"ContainerStarted","Data":"ae710ea597ce9f2e85b7a51a4dbc0dc0c5fbdb9e68ac1ad4496b9f82ca3d3bb3"} Sep 10 12:06:41.994848 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.993386 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" event={"ID":"924a1e16-0dea-4f1d-9ec2-2dc240faff7d","Type":"ContainerStarted","Data":"65f71ae76381c1b82b7a2ede97c81e6925c48293170e1e9f42aba59ca41a97f6"} Sep 10 12:06:41.995583 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.995565 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" event={"ID":"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b","Type":"ContainerStarted","Data":"3a480dbe53abf7df03620e4dc00eb59f05e61436b235b838971b4b4796482b6b"} Sep 10 12:06:41.997329 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.997312 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lg9xf" event={"ID":"227a6eda-79a6-49f9-9bed-eb32fbeff27a","Type":"ContainerStarted","Data":"3362c3a7fb002c54ef72c324d64999c96b15f7eca2e26bb7ab09e33151166fa3"} Sep 10 12:06:41.998720 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:41.998696 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal" event={"ID":"4e79bb2805f50e77c363fc54fc65fe32","Type":"ContainerStarted","Data":"163ab7a9ddda0b7a8b15fb00fb001b4bbbbdaa2e4dd5aa8fad4e6abe5de62b1e"} Sep 10 12:06:42.000088 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:42.000072 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fb9x6" event={"ID":"0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108","Type":"ContainerStarted","Data":"e254e76f2865be6be5a0564192a85d5741f5de8de1a17224da1cb08cf41760c0"} Sep 10 12:06:42.001403 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:42.001385 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" event={"ID":"216eee13-735c-4c33-8377-40dacf2c051d","Type":"ContainerStarted","Data":"c1aee4aacae897b716d141a578b30888394eb9b78959d193aa5f0a10da0b7453"} Sep 10 12:06:42.004331 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:42.004293 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-klmnw" podStartSLOduration=2.817558563 podStartE2EDuration="22.004281515s" totalImagesPullingTime="19.186722952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:42.003730798 +0000 UTC m=+22.581054407" watchObservedRunningTime="2026-09-10 12:06:42.004281515 +0000 UTC m=+22.581605124" Sep 10 12:06:42.023937 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:42.023891 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lg9xf" podStartSLOduration=4.005776879 podStartE2EDuration="23.023877591s" totalImagesPullingTime="19.018100712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:42.023854868 +0000 UTC m=+22.601178476" watchObservedRunningTime="2026-09-10 12:06:42.023877591 +0000 UTC m=+22.601201200" Sep 10 12:06:42.048064 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:42.048017 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xv2fl" podStartSLOduration=2.868849976 podStartE2EDuration="22.048003225s" totalImagesPullingTime="19.179153249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:42.044197338 +0000 UTC m=+22.621520939" watchObservedRunningTime="2026-09-10 12:06:42.048003225 +0000 UTC m=+22.625326833" Sep 10 12:06:42.065169 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:42.065120 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-9.ec2.internal" podStartSLOduration=22.065104371 podStartE2EDuration="22.065104371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:42.065012262 +0000 UTC m=+22.642335881" watchObservedRunningTime="2026-09-10 12:06:42.065104371 +0000 UTC m=+22.642427979" Sep 10 12:06:42.115834 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:42.115776 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bxjqj" podStartSLOduration=3.900895804 podStartE2EDuration="23.115759239s" totalImagesPullingTime="19.214863435s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:42.092856275 +0000 UTC m=+22.670179884" watchObservedRunningTime="2026-09-10 12:06:42.115759239 +0000 UTC m=+22.693082873" Sep 10 12:06:42.896338 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:42.896302 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:42.896592 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:42.896468 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:43.005818 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:43.005712 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mcbp8" event={"ID":"2605e86c-70b6-4188-8c87-4f8629dafbd7","Type":"ContainerStarted","Data":"9336539cd64f4e35d517a38e5c8de8a7ba6a1d75f0f3986f4fe7cffad0b8c1e3"} Sep 10 12:06:43.023699 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:43.023641 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fb9x6" podStartSLOduration=3.7937637029999998 podStartE2EDuration="23.02362571s" totalImagesPullingTime="19.229862007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:42.145275973 +0000 UTC m=+22.722599582" watchObservedRunningTime="2026-09-10 12:06:43.02362571 +0000 UTC m=+23.600949319" Sep 10 12:06:43.023881 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:43.023791 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mcbp8" podStartSLOduration=4.808812373 podStartE2EDuration="24.02378656s" totalImagesPullingTime="19.214974187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:43.023230682 +0000 UTC m=+23.600554292" watchObservedRunningTime="2026-09-10 12:06:43.02378656 +0000 UTC m=+23.601110169" Sep 10 12:06:43.307710 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:43.307519 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 12:06:43.841567 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:43.841457 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T12:06:43.307703439Z","UUID":"0268e515-acc3-46fc-aa4d-e96c3dc4d034","Handler":null,"Name":"","Endpoint":""} Sep 10 12:06:43.842724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:43.842708 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 12:06:43.842825 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:43.842738 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 12:06:43.896245 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:43.896212 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:43.896425 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:43.896224 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:43.896425 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:43.896333 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:43.896425 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:43.896395 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:44.009518 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:44.009490 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" event={"ID":"83631e085fe0c12b5412351d1accf045","Type":"ContainerStarted","Data":"1a77b546a66c346329d6386df14c97123073b9de27f2c82552a85f43e25d75d3"} Sep 10 12:06:44.015301 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:44.015267 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" event={"ID":"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b","Type":"ContainerStarted","Data":"ffa84e49871fbca039a4d63da1f7d838b381d04c3deb2c0ae21a1ad1f297adbe"} Sep 10 12:06:44.895943 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:44.895909 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:44.896170 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:44.896044 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:45.022098 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:45.022058 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" event={"ID":"924a1e16-0dea-4f1d-9ec2-2dc240faff7d","Type":"ContainerStarted","Data":"22ce2d12d377296b16180708e65f44f9a8284add7dbe0a4ce0822f8d3f8aa0ba"} Sep 10 12:06:45.896258 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:45.896220 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:45.896462 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:45.896221 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:45.896462 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:45.896372 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:45.896462 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:45.896455 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:45.956626 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:45.956590 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:45.957400 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:45.957373 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:45.973775 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:45.973715 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-9.ec2.internal" podStartSLOduration=25.973697769 podStartE2EDuration="25.973697769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:44.053563312 +0000 UTC m=+24.630886920" watchObservedRunningTime="2026-09-10 12:06:45.973697769 +0000 UTC m=+26.551021379" Sep 10 12:06:46.027287 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:46.027246 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" event={"ID":"7b836b1e-a88e-4d6b-8c71-6a0fc025b52b","Type":"ContainerStarted","Data":"1788d3b6384d0ab98fd656f599e7e10d4e35d64d155d93717940a23632fc6064"} Sep 10 12:06:46.027724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:46.027489 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:46.027975 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:46.027955 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lg9xf" Sep 10 12:06:46.049493 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:46.049435 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tqc5" podStartSLOduration=3.9401897679999998 podStartE2EDuration="26.049416115s" totalImagesPullingTime="22.109226347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:46.048944632 +0000 UTC m=+26.626268242" watchObservedRunningTime="2026-09-10 12:06:46.049416115 +0000 UTC m=+26.626739724" Sep 10 12:06:46.763130 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:46.763112 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-klmnw_dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3/dns-node-resolver/0.log" Sep 10 12:06:46.896584 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:46.896415 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:46.896758 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:46.896687 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:47.031634 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:47.031601 2583 generic.go:356] "Generic (PLEG): container finished" podID="31074e24-3342-4c93-94a8-9abc7e0e454e" containerID="038c297864359d0ab589c59290cf7687f0cabd98d1fbd6d1e39ce46f42807f4c" exitCode=0 Sep 10 12:06:47.032220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:47.031656 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" event={"ID":"31074e24-3342-4c93-94a8-9abc7e0e454e","Type":"ContainerDied","Data":"038c297864359d0ab589c59290cf7687f0cabd98d1fbd6d1e39ce46f42807f4c"} Sep 10 12:06:47.039288 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:47.039258 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" event={"ID":"924a1e16-0dea-4f1d-9ec2-2dc240faff7d","Type":"ContainerStarted","Data":"18208b702438c6404f81f9327d10183d6206b2e5040e17b5461c90c5aa960a95"} Sep 10 12:06:47.123018 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:47.122967 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" podStartSLOduration=7.948333816 podStartE2EDuration="27.122950093s" totalImagesPullingTime="19.174616277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:47.118083714 +0000 UTC m=+27.695407334" watchObservedRunningTime="2026-09-10 12:06:47.122950093 +0000 UTC m=+27.700273701" Sep 10 12:06:47.151806 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:47.151741 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bxjqj_1dfb8b69-72b9-4fb8-b33d-a2b938b653a1/node-ca/0.log" Sep 10 12:06:47.896516 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:47.896480 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:47.897063 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:47.896610 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:47.897312 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:47.897155 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:47.897312 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:47.897270 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:48.042504 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.042476 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:48.042918 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.042559 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:48.042918 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.042609 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:48.061884 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.061851 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:48.062264 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.062249 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:06:48.597146 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.597112 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7wxnx"] Sep 10 12:06:48.597331 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.597268 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:48.597437 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:48.597371 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:48.603465 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.603442 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9wlph"] Sep 10 12:06:48.603589 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.603547 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:48.603659 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:48.603637 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:48.603965 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.603946 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s8hwl"] Sep 10 12:06:48.604075 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:48.604052 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:48.604191 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:48.604152 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:49.045500 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:49.045465 2583 generic.go:356] "Generic (PLEG): container finished" podID="31074e24-3342-4c93-94a8-9abc7e0e454e" containerID="4f5c7814d1b8868734179bcff290bc6a6a3591a33f37478246b4e27fc8f8ec6f" exitCode=0 Sep 10 12:06:49.046283 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:49.045556 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" event={"ID":"31074e24-3342-4c93-94a8-9abc7e0e454e","Type":"ContainerDied","Data":"4f5c7814d1b8868734179bcff290bc6a6a3591a33f37478246b4e27fc8f8ec6f"} Sep 10 12:06:49.896832 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:49.896643 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:49.897025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:49.896712 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:49.897025 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:49.896904 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:49.897025 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:49.896983 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:50.896019 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:50.895985 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:50.896419 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:50.896110 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:51.895871 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:51.895831 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:51.896073 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:51.895833 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:51.896073 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:51.895970 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:51.896073 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:51.896004 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:52.055466 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:52.055435 2583 generic.go:356] "Generic (PLEG): container finished" podID="31074e24-3342-4c93-94a8-9abc7e0e454e" containerID="c2b8cc0bd5fda1e8cc4df221e1e1b59f8bce36d42f92f64652dd698c6c9e51ad" exitCode=0 Sep 10 12:06:52.055607 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:52.055496 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" event={"ID":"31074e24-3342-4c93-94a8-9abc7e0e454e","Type":"ContainerDied","Data":"c2b8cc0bd5fda1e8cc4df221e1e1b59f8bce36d42f92f64652dd698c6c9e51ad"} Sep 10 12:06:52.896071 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:52.896037 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:52.896516 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:52.896204 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:53.517627 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:53.517585 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:53.517958 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.517767 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:53.517958 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.517926 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs podName:35e290b1-c39f-4ab8-b6ed-9584e49b3ae4 nodeName:}" failed. No retries permitted until 2026-09-10 12:07:25.517901911 +0000 UTC m=+66.095225499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs") pod "network-metrics-daemon-s8hwl" (UID: "35e290b1-c39f-4ab8-b6ed-9584e49b3ae4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:06:53.619047 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:53.619006 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:53.619231 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.619144 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:53.619231 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.619220 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret podName:faf4ab54-b2cc-42bf-a906-e23f8d7949d9 nodeName:}" failed. No retries permitted until 2026-09-10 12:07:25.619200107 +0000 UTC m=+66.196523708 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret") pod "global-pull-secret-syncer-7wxnx" (UID: "faf4ab54-b2cc-42bf-a906-e23f8d7949d9") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:06:53.720120 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:53.720068 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:53.720290 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.720228 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:06:53.720290 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.720247 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:06:53.720290 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.720257 2583 projected.go:196] Error preparing data for projected volume kube-api-access-gb2ck for pod openshift-network-diagnostics/network-check-target-9wlph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:53.720444 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.720303 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck podName:3323f642-5184-411e-8db6-d639e9259107 nodeName:}" failed. No retries permitted until 2026-09-10 12:07:25.720286085 +0000 UTC m=+66.297609672 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gb2ck" (UniqueName: "kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck") pod "network-check-target-9wlph" (UID: "3323f642-5184-411e-8db6-d639e9259107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:06:53.896486 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:53.896454 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:53.896952 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:53.896455 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:53.896952 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.896581 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:53.896952 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:53.896667 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:54.896742 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:54.896704 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:54.897192 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:54.896833 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:55.896525 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:55.896486 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:55.896699 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:55.896486 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:55.896699 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:55.896617 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:55.897080 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:55.896714 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:56.896818 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:56.896781 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:56.897235 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:56.896902 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:57.896758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:57.896723 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:57.896889 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:57.896774 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:57.896889 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:57.896864 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:57.897210 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:57.896996 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:06:58.073139 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:58.072953 2583 generic.go:356] "Generic (PLEG): container finished" podID="31074e24-3342-4c93-94a8-9abc7e0e454e" containerID="f35d4351496ae7ecd94d045ec7f05c09f99d3fce04efc15f132b3ff9d7a1a469" exitCode=0 Sep 10 12:06:58.073301 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:58.073019 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" event={"ID":"31074e24-3342-4c93-94a8-9abc7e0e454e","Type":"ContainerDied","Data":"f35d4351496ae7ecd94d045ec7f05c09f99d3fce04efc15f132b3ff9d7a1a469"} Sep 10 12:06:58.895936 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:58.895902 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:06:58.896122 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:58.896012 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:06:59.079027 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:59.078997 2583 generic.go:356] "Generic (PLEG): container finished" podID="31074e24-3342-4c93-94a8-9abc7e0e454e" containerID="3bb0ffbca13b0f8ed58337689a6fcbc36c1ff32c8d0b22c4ab43afb4c6477893" exitCode=0 Sep 10 12:06:59.079404 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:59.079056 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" event={"ID":"31074e24-3342-4c93-94a8-9abc7e0e454e","Type":"ContainerDied","Data":"3bb0ffbca13b0f8ed58337689a6fcbc36c1ff32c8d0b22c4ab43afb4c6477893"} Sep 10 12:06:59.896985 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:59.896931 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:06:59.897192 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:59.897057 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:06:59.897192 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:06:59.897103 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:06:59.897314 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:06:59.897189 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:07:00.084887 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:00.084855 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" event={"ID":"31074e24-3342-4c93-94a8-9abc7e0e454e","Type":"ContainerStarted","Data":"2cb05868c31fa7344cfec8ce07a0d4e43ee5180ed4f18c5c10902c6eacaa1ffa"} Sep 10 12:07:00.895842 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:00.895802 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:00.896016 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:00.895917 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:07:01.896164 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:01.896128 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:01.896673 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:01.896137 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:07:01.896673 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:01.896230 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:07:01.896673 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:01.896298 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:07:02.896449 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:02.896416 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:02.896878 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:02.896548 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:07:03.896696 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:03.896652 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:03.897113 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:03.896667 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:07:03.897113 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:03.896763 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:07:03.897113 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:03.896844 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:07:04.895826 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:04.895793 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:04.895997 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:04.895903 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:07:05.896073 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:05.896030 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:05.896073 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:05.896074 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:07:05.896516 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:05.896157 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:07:05.896516 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:05.896275 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:07:06.896328 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:06.896284 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:06.896804 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:06.896452 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:07:07.896390 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:07.896337 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:07:07.896765 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:07.896346 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:07.896765 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:07.896470 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:07:07.896765 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:07.896530 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:07:08.895804 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:08.895772 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:08.895978 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:08.895877 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:07:09.896576 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:09.896538 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:07:09.897142 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:09.896618 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wxnx" podUID="faf4ab54-b2cc-42bf-a906-e23f8d7949d9" Sep 10 12:07:09.897142 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:09.896713 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:09.897142 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:09.896790 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wlph" podUID="3323f642-5184-411e-8db6-d639e9259107" Sep 10 12:07:10.895827 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:10.895794 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:10.895979 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:10.895900 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8hwl" podUID="35e290b1-c39f-4ab8-b6ed-9584e49b3ae4" Sep 10 12:07:11.209797 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.209718 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-9.ec2.internal" event="NodeReady" Sep 10 12:07:11.210203 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.209837 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 12:07:11.280794 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.280713 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vsdv2" podStartSLOduration=19.671478484 podStartE2EDuration="52.280693678s" totalImagesPullingTime="32.609215194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:00.114704537 +0000 UTC m=+40.692028147" watchObservedRunningTime="2026-09-10 12:07:11.280693678 +0000 UTC m=+51.858017284" Sep 10 12:07:11.280987 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.280970 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bkljw"] Sep 10 12:07:11.285378 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.285343 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.289168 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.289145 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bp4gw\"" Sep 10 12:07:11.289582 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.289558 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 12:07:11.289582 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.289577 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 12:07:11.289745 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.289564 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nsznl"] Sep 10 12:07:11.290034 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.290020 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 12:07:11.293733 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.293712 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nsznl" Sep 10 12:07:11.295736 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.295715 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 12:07:11.295953 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.295937 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 12:07:11.296037 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.296019 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 12:07:11.296037 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.296027 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tf7wk\"" Sep 10 12:07:11.296278 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.296264 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 12:07:11.303669 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.303649 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bkljw"] Sep 10 12:07:11.308278 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.308257 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nsznl"] Sep 10 12:07:11.383461 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.383428 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-55t8l"] Sep 10 12:07:11.387515 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.387501 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.389625 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.389607 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 12:07:11.389950 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.389930 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 12:07:11.390060 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.390045 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4wjzz\"" Sep 10 12:07:11.404060 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.404036 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-55t8l"] Sep 10 12:07:11.438292 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.438257 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a368f756-d49b-4873-8ae0-108960413ac6-cert\") pod \"ingress-canary-nsznl\" (UID: \"a368f756-d49b-4873-8ae0-108960413ac6\") " pod="openshift-ingress-canary/ingress-canary-nsznl" Sep 10 12:07:11.438292 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.438296 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52d6c263-5574-4700-81b6-e4d979cc71c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.438527 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.438320 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx798\" (UniqueName: \"kubernetes.io/projected/52d6c263-5574-4700-81b6-e4d979cc71c3-kube-api-access-lx798\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.438527 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.438337 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/52d6c263-5574-4700-81b6-e4d979cc71c3-data-volume\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.438527 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.438437 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/52d6c263-5574-4700-81b6-e4d979cc71c3-crio-socket\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.438527 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.438474 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52d6c263-5574-4700-81b6-e4d979cc71c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.438527 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.438505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj57w\" (UniqueName: \"kubernetes.io/projected/a368f756-d49b-4873-8ae0-108960413ac6-kube-api-access-hj57w\") pod \"ingress-canary-nsznl\" (UID: \"a368f756-d49b-4873-8ae0-108960413ac6\") " pod="openshift-ingress-canary/ingress-canary-nsznl" Sep 10 12:07:11.539235 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539206 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-metrics-tls\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.539407 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539239 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5b9x\" (UniqueName: \"kubernetes.io/projected/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-kube-api-access-m5b9x\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.539407 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539264 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-config-volume\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.539407 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539327 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a368f756-d49b-4873-8ae0-108960413ac6-cert\") pod \"ingress-canary-nsznl\" (UID: \"a368f756-d49b-4873-8ae0-108960413ac6\") " pod="openshift-ingress-canary/ingress-canary-nsznl" Sep 10 12:07:11.539407 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539385 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52d6c263-5574-4700-81b6-e4d979cc71c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.539580 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539414 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx798\" (UniqueName: \"kubernetes.io/projected/52d6c263-5574-4700-81b6-e4d979cc71c3-kube-api-access-lx798\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.539580 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539433 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-tmp-dir\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.539580 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539467 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/52d6c263-5574-4700-81b6-e4d979cc71c3-data-volume\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.539580 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539515 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/52d6c263-5574-4700-81b6-e4d979cc71c3-crio-socket\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.539580 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539542 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52d6c263-5574-4700-81b6-e4d979cc71c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.539580 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj57w\" (UniqueName: \"kubernetes.io/projected/a368f756-d49b-4873-8ae0-108960413ac6-kube-api-access-hj57w\") pod \"ingress-canary-nsznl\" (UID: \"a368f756-d49b-4873-8ae0-108960413ac6\") " pod="openshift-ingress-canary/ingress-canary-nsznl" Sep 10 12:07:11.539966 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.539944 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/52d6c263-5574-4700-81b6-e4d979cc71c3-crio-socket\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.543167 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.540131 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/52d6c263-5574-4700-81b6-e4d979cc71c3-data-volume\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.543167 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.540383 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52d6c263-5574-4700-81b6-e4d979cc71c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.545094 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.545059 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52d6c263-5574-4700-81b6-e4d979cc71c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.545426 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.545409 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a368f756-d49b-4873-8ae0-108960413ac6-cert\") pod \"ingress-canary-nsznl\" (UID: \"a368f756-d49b-4873-8ae0-108960413ac6\") " pod="openshift-ingress-canary/ingress-canary-nsznl" Sep 10 12:07:11.549368 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.549322 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj57w\" (UniqueName: \"kubernetes.io/projected/a368f756-d49b-4873-8ae0-108960413ac6-kube-api-access-hj57w\") pod \"ingress-canary-nsznl\" (UID: \"a368f756-d49b-4873-8ae0-108960413ac6\") " pod="openshift-ingress-canary/ingress-canary-nsznl" Sep 10 12:07:11.550196 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.550174 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx798\" (UniqueName: \"kubernetes.io/projected/52d6c263-5574-4700-81b6-e4d979cc71c3-kube-api-access-lx798\") pod \"insights-runtime-extractor-bkljw\" (UID: \"52d6c263-5574-4700-81b6-e4d979cc71c3\") " pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.594372 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.594287 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bkljw" Sep 10 12:07:11.602032 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.602012 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nsznl" Sep 10 12:07:11.640973 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.640319 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-metrics-tls\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.640973 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.640387 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5b9x\" (UniqueName: \"kubernetes.io/projected/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-kube-api-access-m5b9x\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.640973 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.640429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-config-volume\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.640973 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.640473 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-tmp-dir\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.640973 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.640828 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-tmp-dir\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.641380 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.641188 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-config-volume\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.642819 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.642779 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-metrics-tls\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.650461 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.650097 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5b9x\" (UniqueName: \"kubernetes.io/projected/f15ce4a4-4f2e-4b78-9c48-3c169c83439a-kube-api-access-m5b9x\") pod \"dns-default-55t8l\" (UID: \"f15ce4a4-4f2e-4b78-9c48-3c169c83439a\") " pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.695886 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.695844 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:11.893434 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.893308 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nsznl"] Sep 10 12:07:11.896189 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.896113 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:11.896608 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.896541 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:07:11.898534 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:11.898508 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda368f756_d49b_4873_8ae0_108960413ac6.slice/crio-14656da0733ef800b6ce8c7f79bbf7aca063e3c7e2601132ba89738838dab863 WatchSource:0}: Error finding container 14656da0733ef800b6ce8c7f79bbf7aca063e3c7e2601132ba89738838dab863: Status 404 returned error can't find the container with id 14656da0733ef800b6ce8c7f79bbf7aca063e3c7e2601132ba89738838dab863 Sep 10 12:07:11.899795 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.899214 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 12:07:11.899795 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.899373 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 12:07:11.899795 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.899458 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 12:07:11.899795 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.899680 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vr5lb\"" Sep 10 12:07:11.900732 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.900709 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bkljw"] Sep 10 12:07:11.901717 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:11.901699 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d6c263_5574_4700_81b6_e4d979cc71c3.slice/crio-cd40d6fc3bad284be49b9d7cb6795dcd3a7d9eeadfbb11919c5863426bb9a13b WatchSource:0}: Error finding container cd40d6fc3bad284be49b9d7cb6795dcd3a7d9eeadfbb11919c5863426bb9a13b: Status 404 returned error can't find the container with id cd40d6fc3bad284be49b9d7cb6795dcd3a7d9eeadfbb11919c5863426bb9a13b Sep 10 12:07:11.976449 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:11.976376 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-55t8l"] Sep 10 12:07:11.981458 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:11.981428 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15ce4a4_4f2e_4b78_9c48_3c169c83439a.slice/crio-0dcb235aa897cf04b1a07ee6aca1f4619861c0144eb58e59c2dc5f69e653006e WatchSource:0}: Error finding container 0dcb235aa897cf04b1a07ee6aca1f4619861c0144eb58e59c2dc5f69e653006e: Status 404 returned error can't find the container with id 0dcb235aa897cf04b1a07ee6aca1f4619861c0144eb58e59c2dc5f69e653006e Sep 10 12:07:12.111671 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:12.111634 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nsznl" event={"ID":"a368f756-d49b-4873-8ae0-108960413ac6","Type":"ContainerStarted","Data":"14656da0733ef800b6ce8c7f79bbf7aca063e3c7e2601132ba89738838dab863"} Sep 10 12:07:12.113110 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:12.113088 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkljw" event={"ID":"52d6c263-5574-4700-81b6-e4d979cc71c3","Type":"ContainerStarted","Data":"1c774dffcc8a1af58c2e47e0032a2a7930dba4c8e887abed9605af4bc082620c"} Sep 10 12:07:12.113262 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:12.113139 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkljw" event={"ID":"52d6c263-5574-4700-81b6-e4d979cc71c3","Type":"ContainerStarted","Data":"cd40d6fc3bad284be49b9d7cb6795dcd3a7d9eeadfbb11919c5863426bb9a13b"} Sep 10 12:07:12.114197 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:12.114173 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-55t8l" event={"ID":"f15ce4a4-4f2e-4b78-9c48-3c169c83439a","Type":"ContainerStarted","Data":"0dcb235aa897cf04b1a07ee6aca1f4619861c0144eb58e59c2dc5f69e653006e"} Sep 10 12:07:12.896579 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:12.896544 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:12.899889 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:12.899866 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 12:07:12.900008 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:12.899923 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-llnv7\"" Sep 10 12:07:14.121095 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:14.121053 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkljw" event={"ID":"52d6c263-5574-4700-81b6-e4d979cc71c3","Type":"ContainerStarted","Data":"43b7ef927451ce6b0319dec806e9163c955d2403aac83f3f853760c3767f93d5"} Sep 10 12:07:16.129810 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:16.129766 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-55t8l" event={"ID":"f15ce4a4-4f2e-4b78-9c48-3c169c83439a","Type":"ContainerStarted","Data":"09731e2f62c53c2feedda107231ce7b743fd40ada992e138d9ff5ea638879ac7"} Sep 10 12:07:16.131516 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:16.131485 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nsznl" event={"ID":"a368f756-d49b-4873-8ae0-108960413ac6","Type":"ContainerStarted","Data":"8e7563eb475be7bc3f303dbbf05fb5fa5086c7780a2761af980e9ff02b23b06d"} Sep 10 12:07:16.151922 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:16.151854 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nsznl" podStartSLOduration=1.117776084 podStartE2EDuration="5.151831016s" totalImagesPullingTime="4.034054932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:16.150691669 +0000 UTC m=+56.728015279" watchObservedRunningTime="2026-09-10 12:07:16.151831016 +0000 UTC m=+56.729154626" Sep 10 12:07:17.136585 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.136548 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-55t8l" event={"ID":"f15ce4a4-4f2e-4b78-9c48-3c169c83439a","Type":"ContainerStarted","Data":"89db9ab28345791c23f94830b2d25c1cc4ffa7e04ae2f9dda11ac2016d6dd791"} Sep 10 12:07:17.137042 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.136666 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:17.138631 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.138608 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkljw" event={"ID":"52d6c263-5574-4700-81b6-e4d979cc71c3","Type":"ContainerStarted","Data":"00bb71d17cccf273e7f0c4fe486bfc99637f77bcff5c4650e22eb9f99b4d6a0f"} Sep 10 12:07:17.181099 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.181046 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-55t8l" podStartSLOduration=2.229678135 podStartE2EDuration="6.181027361s" totalImagesPullingTime="3.951349226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:17.157160213 +0000 UTC m=+57.734483821" watchObservedRunningTime="2026-09-10 12:07:17.181027361 +0000 UTC m=+57.758350971" Sep 10 12:07:17.181509 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.181481 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bkljw" podStartSLOduration=1.8084734770000002 podStartE2EDuration="6.181471249s" totalImagesPullingTime="4.372997772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:07:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:17.180818908 +0000 UTC m=+57.758142518" watchObservedRunningTime="2026-09-10 12:07:17.181471249 +0000 UTC m=+57.758794858" Sep 10 12:07:17.604644 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.604597 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-6ftmg"] Sep 10 12:07:17.607496 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.607480 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-6ftmg" Sep 10 12:07:17.610617 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.610596 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jkjd8\"" Sep 10 12:07:17.610682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.610651 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 12:07:17.611188 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.611174 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 12:07:17.623839 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.623806 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-6ftmg"] Sep 10 12:07:17.785313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.785252 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4khg\" (UniqueName: \"kubernetes.io/projected/cbbf08dd-74e3-476f-89f1-a883cb9b5054-kube-api-access-n4khg\") pod \"downloads-85d98c998f-6ftmg\" (UID: \"cbbf08dd-74e3-476f-89f1-a883cb9b5054\") " pod="openshift-console/downloads-85d98c998f-6ftmg" Sep 10 12:07:17.885937 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.885858 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4khg\" (UniqueName: \"kubernetes.io/projected/cbbf08dd-74e3-476f-89f1-a883cb9b5054-kube-api-access-n4khg\") pod \"downloads-85d98c998f-6ftmg\" (UID: \"cbbf08dd-74e3-476f-89f1-a883cb9b5054\") " pod="openshift-console/downloads-85d98c998f-6ftmg" Sep 10 12:07:17.903006 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.902978 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4khg\" (UniqueName: \"kubernetes.io/projected/cbbf08dd-74e3-476f-89f1-a883cb9b5054-kube-api-access-n4khg\") pod \"downloads-85d98c998f-6ftmg\" (UID: \"cbbf08dd-74e3-476f-89f1-a883cb9b5054\") " pod="openshift-console/downloads-85d98c998f-6ftmg" Sep 10 12:07:17.916007 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:17.915977 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-6ftmg" Sep 10 12:07:18.162285 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.162201 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-2lg8p"] Sep 10 12:07:18.166130 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.166112 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.171176 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.171156 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 10 12:07:18.171608 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.171587 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dlgdr\"" Sep 10 12:07:18.171608 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.171620 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 10 12:07:18.171888 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.171734 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 12:07:18.171888 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.171748 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 12:07:18.171888 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.171855 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 12:07:18.176738 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.176719 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-2lg8p"] Sep 10 12:07:18.194806 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.194782 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-6ftmg"] Sep 10 12:07:18.198015 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:18.197993 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbbf08dd_74e3_476f_89f1_a883cb9b5054.slice/crio-fa25b76c0325189d98df77850db6b65f5e4693786079a4670a9822c729af3ec8 WatchSource:0}: Error finding container fa25b76c0325189d98df77850db6b65f5e4693786079a4670a9822c729af3ec8: Status 404 returned error can't find the container with id fa25b76c0325189d98df77850db6b65f5e4693786079a4670a9822c729af3ec8 Sep 10 12:07:18.289573 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.289527 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8754300-e471-407b-bf99-0f7663363762-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.289573 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.289580 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8754300-e471-407b-bf99-0f7663363762-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.289826 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.289668 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hd8h\" (UniqueName: \"kubernetes.io/projected/a8754300-e471-407b-bf99-0f7663363762-kube-api-access-5hd8h\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.289888 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.289810 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8754300-e471-407b-bf99-0f7663363762-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.391058 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.390952 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hd8h\" (UniqueName: \"kubernetes.io/projected/a8754300-e471-407b-bf99-0f7663363762-kube-api-access-5hd8h\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.391240 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.391120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8754300-e471-407b-bf99-0f7663363762-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.391240 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.391188 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8754300-e471-407b-bf99-0f7663363762-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.391240 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.391222 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8754300-e471-407b-bf99-0f7663363762-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.391407 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:18.391266 2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 10 12:07:18.391407 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:18.391326 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8754300-e471-407b-bf99-0f7663363762-prometheus-operator-tls podName:a8754300-e471-407b-bf99-0f7663363762 nodeName:}" failed. No retries permitted until 2026-09-10 12:07:18.891308088 +0000 UTC m=+59.468631679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a8754300-e471-407b-bf99-0f7663363762-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-2lg8p" (UID: "a8754300-e471-407b-bf99-0f7663363762") : secret "prometheus-operator-tls" not found Sep 10 12:07:18.391862 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.391842 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8754300-e471-407b-bf99-0f7663363762-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.393792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.393775 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8754300-e471-407b-bf99-0f7663363762-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.408395 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.408344 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hd8h\" (UniqueName: \"kubernetes.io/projected/a8754300-e471-407b-bf99-0f7663363762-kube-api-access-5hd8h\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.893596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.893553 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8754300-e471-407b-bf99-0f7663363762-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:18.896049 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:18.896017 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8754300-e471-407b-bf99-0f7663363762-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2lg8p\" (UID: \"a8754300-e471-407b-bf99-0f7663363762\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:19.075703 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:19.075667 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" Sep 10 12:07:19.150108 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:19.150069 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-6ftmg" event={"ID":"cbbf08dd-74e3-476f-89f1-a883cb9b5054","Type":"ContainerStarted","Data":"fa25b76c0325189d98df77850db6b65f5e4693786079a4670a9822c729af3ec8"} Sep 10 12:07:19.376372 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:19.376312 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-2lg8p"] Sep 10 12:07:19.380220 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:19.380192 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8754300_e471_407b_bf99_0f7663363762.slice/crio-8bfdc2e5df56818f3667038cb3c22f56afd5429f352fd356c3061c4e962c9014 WatchSource:0}: Error finding container 8bfdc2e5df56818f3667038cb3c22f56afd5429f352fd356c3061c4e962c9014: Status 404 returned error can't find the container with id 8bfdc2e5df56818f3667038cb3c22f56afd5429f352fd356c3061c4e962c9014 Sep 10 12:07:20.067677 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:20.067643 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2znnx" Sep 10 12:07:20.156501 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:20.156452 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" event={"ID":"a8754300-e471-407b-bf99-0f7663363762","Type":"ContainerStarted","Data":"8bfdc2e5df56818f3667038cb3c22f56afd5429f352fd356c3061c4e962c9014"} Sep 10 12:07:21.162168 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:21.162122 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" event={"ID":"a8754300-e471-407b-bf99-0f7663363762","Type":"ContainerStarted","Data":"b1bf9f575bbad67b27748747742d09934a0a3c8f6b2e2165c6a71ac57419d58f"} Sep 10 12:07:21.162707 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:21.162184 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" event={"ID":"a8754300-e471-407b-bf99-0f7663363762","Type":"ContainerStarted","Data":"9d8aa0458a61bfb8511302b88d0edd9c1e26f87e29af9b5d2cafa530c5fbfc61"} Sep 10 12:07:21.194453 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:21.191913 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-2lg8p" podStartSLOduration=1.651448518 podStartE2EDuration="3.19189362s" totalImagesPullingTime="1.540445102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:21.19154819 +0000 UTC m=+61.768871800" watchObservedRunningTime="2026-09-10 12:07:21.19189362 +0000 UTC m=+61.769217247" Sep 10 12:07:23.664157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.664120 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v"] Sep 10 12:07:23.667459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.667435 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-t5j77"] Sep 10 12:07:23.667611 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.667577 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.669868 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.669843 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 10 12:07:23.670374 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.670337 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 10 12:07:23.672771 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.670535 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wn67m\"" Sep 10 12:07:23.672771 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.671255 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.677971 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.677611 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 10 12:07:23.677971 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.677649 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-f76x5\"" Sep 10 12:07:23.677971 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.677796 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 10 12:07:23.678195 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.678172 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 10 12:07:23.700654 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.700616 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v"] Sep 10 12:07:23.700809 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.700698 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-t5j77"] Sep 10 12:07:23.700854 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.700819 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qcjkz"] Sep 10 12:07:23.704329 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.704301 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.706280 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.706133 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 12:07:23.706280 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.706261 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qv5br\"" Sep 10 12:07:23.706609 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.706592 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 12:07:23.706683 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.706665 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 12:07:23.834223 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834185 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-textfile\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.834223 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834232 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3510d8b5-99e5-4726-a1f3-177d939fe640-root\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.834582 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834263 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.834582 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834330 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-wtmp\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.834582 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834417 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3510d8b5-99e5-4726-a1f3-177d939fe640-metrics-client-ca\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.834582 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834470 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.834582 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834513 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-tls\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.834582 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834571 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtcpj\" (UniqueName: \"kubernetes.io/projected/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-kube-api-access-dtcpj\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.834881 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834597 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.834881 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.834881 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834655 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3510d8b5-99e5-4726-a1f3-177d939fe640-sys\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.834881 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2d5d9d44-6619-4706-9689-b66c12635aa6-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.834881 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834725 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d5d9d44-6619-4706-9689-b66c12635aa6-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.834881 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834798 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.834881 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834827 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.834881 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834853 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-accelerators-collector-config\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.835211 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834895 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzw9p\" (UniqueName: \"kubernetes.io/projected/3510d8b5-99e5-4726-a1f3-177d939fe640-kube-api-access-nzw9p\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.835211 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834922 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.835211 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.834945 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8rvk\" (UniqueName: \"kubernetes.io/projected/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-api-access-m8rvk\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.936204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936110 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3510d8b5-99e5-4726-a1f3-177d939fe640-metrics-client-ca\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.936204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936172 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.936477 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936300 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-tls\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.936477 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtcpj\" (UniqueName: \"kubernetes.io/projected/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-kube-api-access-dtcpj\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.936477 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936396 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.936477 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936434 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.936477 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936459 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3510d8b5-99e5-4726-a1f3-177d939fe640-sys\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.936724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936488 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2d5d9d44-6619-4706-9689-b66c12635aa6-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.936724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d5d9d44-6619-4706-9689-b66c12635aa6-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.936724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.936724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936597 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.936724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936621 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-accelerators-collector-config\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.936724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzw9p\" (UniqueName: \"kubernetes.io/projected/3510d8b5-99e5-4726-a1f3-177d939fe640-kube-api-access-nzw9p\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.936724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936687 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.936724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936713 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8rvk\" (UniqueName: \"kubernetes.io/projected/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-api-access-m8rvk\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.937116 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-textfile\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.937116 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936767 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3510d8b5-99e5-4726-a1f3-177d939fe640-root\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.937116 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.937116 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936817 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-wtmp\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.937116 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936889 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3510d8b5-99e5-4726-a1f3-177d939fe640-metrics-client-ca\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.937116 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936968 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2d5d9d44-6619-4706-9689-b66c12635aa6-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.937116 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.936993 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-wtmp\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.937659 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.937337 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-accelerators-collector-config\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.938091 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.938060 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.938190 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.938130 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.938669 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.938629 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-textfile\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.938669 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.938653 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d5d9d44-6619-4706-9689-b66c12635aa6-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.938824 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.938715 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3510d8b5-99e5-4726-a1f3-177d939fe640-root\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.938824 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.938762 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3510d8b5-99e5-4726-a1f3-177d939fe640-sys\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.939430 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.939413 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.939627 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.939593 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.940368 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.940323 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.941054 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.941024 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-tls\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.941226 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.941205 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3510d8b5-99e5-4726-a1f3-177d939fe640-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.942672 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.942651 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.946085 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.946063 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzw9p\" (UniqueName: \"kubernetes.io/projected/3510d8b5-99e5-4726-a1f3-177d939fe640-kube-api-access-nzw9p\") pod \"node-exporter-qcjkz\" (UID: \"3510d8b5-99e5-4726-a1f3-177d939fe640\") " pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:23.950055 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.950032 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtcpj\" (UniqueName: \"kubernetes.io/projected/152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab-kube-api-access-dtcpj\") pod \"openshift-state-metrics-657fd6c54-fvs6v\" (UID: \"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.952839 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.952819 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8rvk\" (UniqueName: \"kubernetes.io/projected/2d5d9d44-6619-4706-9689-b66c12635aa6-kube-api-access-m8rvk\") pod \"kube-state-metrics-74cb57df7-t5j77\" (UID: \"2d5d9d44-6619-4706-9689-b66c12635aa6\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:23.983692 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.983654 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" Sep 10 12:07:23.990108 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:23.990082 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" Sep 10 12:07:24.015624 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.015412 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qcjkz" Sep 10 12:07:24.032698 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:24.032277 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3510d8b5_99e5_4726_a1f3_177d939fe640.slice/crio-87af86e47fe5a988ed9bf854842eb35a1afe84b7e9f72c5008d2cd8829a86fa1 WatchSource:0}: Error finding container 87af86e47fe5a988ed9bf854842eb35a1afe84b7e9f72c5008d2cd8829a86fa1: Status 404 returned error can't find the container with id 87af86e47fe5a988ed9bf854842eb35a1afe84b7e9f72c5008d2cd8829a86fa1 Sep 10 12:07:24.173724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.173645 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcjkz" event={"ID":"3510d8b5-99e5-4726-a1f3-177d939fe640","Type":"ContainerStarted","Data":"87af86e47fe5a988ed9bf854842eb35a1afe84b7e9f72c5008d2cd8829a86fa1"} Sep 10 12:07:24.284812 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.284777 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v"] Sep 10 12:07:24.287839 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:24.287812 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod152f4948_cfae_4ed7_8ca8_4cb3caa6c6ab.slice/crio-369fd077b50605024ac51a39c3426f2edc01f275a3bb14846cdb9a5f89bf1eea WatchSource:0}: Error finding container 369fd077b50605024ac51a39c3426f2edc01f275a3bb14846cdb9a5f89bf1eea: Status 404 returned error can't find the container with id 369fd077b50605024ac51a39c3426f2edc01f275a3bb14846cdb9a5f89bf1eea Sep 10 12:07:24.292427 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.292401 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-t5j77"] Sep 10 12:07:24.297020 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:24.296997 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5d9d44_6619_4706_9689_b66c12635aa6.slice/crio-951315eb57a0241262066604dbd15d026d92b9cf10b3f78a0daa7889c85ed99f WatchSource:0}: Error finding container 951315eb57a0241262066604dbd15d026d92b9cf10b3f78a0daa7889c85ed99f: Status 404 returned error can't find the container with id 951315eb57a0241262066604dbd15d026d92b9cf10b3f78a0daa7889c85ed99f Sep 10 12:07:24.864805 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.864766 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:07:24.868538 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.868513 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:24.877108 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.877047 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 12:07:24.877199 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.877178 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 12:07:24.878048 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.878029 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 12:07:24.878169 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.878083 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 12:07:24.878228 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.878190 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 12:07:24.878389 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.878375 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 12:07:24.878432 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.878396 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 12:07:24.878951 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.878723 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 12:07:24.878951 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.878723 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 12:07:24.878951 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.878773 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6r9xp\"" Sep 10 12:07:24.913099 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:24.913046 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:07:25.044775 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.044738 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-out\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.044958 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.044786 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.044958 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.044862 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.044958 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.044897 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.044958 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.044924 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.045217 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.044966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.045217 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.044996 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.045217 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.045058 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.045217 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.045123 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-volume\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.045217 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.045166 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-web-config\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.045217 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.045203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.045530 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.045230 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.045530 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.045255 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx4sj\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-kube-api-access-hx4sj\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.146575 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146478 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-out\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.146575 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146537 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.146792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146582 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.146792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146616 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.146792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146644 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.146792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146672 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.146792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146700 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.146792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146729 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.146792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146785 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-volume\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.147123 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146803 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-web-config\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.147123 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146834 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.147123 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146848 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.147123 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.146871 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx4sj\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-kube-api-access-hx4sj\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.147273 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.147224 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.148596 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.148246 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.150326 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.150289 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-out\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.150477 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.150451 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.150593 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.150568 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-web-config\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.150975 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.150950 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.151139 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.151115 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.151581 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.151543 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.151741 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.151664 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.152821 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.152635 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.153042 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.153024 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.154053 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.154029 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-volume\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.158093 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.158065 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx4sj\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-kube-api-access-hx4sj\") pod \"alertmanager-main-0\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.179574 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.179544 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:07:25.179747 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.179662 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" event={"ID":"2d5d9d44-6619-4706-9689-b66c12635aa6","Type":"ContainerStarted","Data":"951315eb57a0241262066604dbd15d026d92b9cf10b3f78a0daa7889c85ed99f"} Sep 10 12:07:25.182064 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.182029 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" event={"ID":"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab","Type":"ContainerStarted","Data":"64bba1c6aa59c85f98051fdc3e880edc17f2e52f1735f894d6d57af79dc04204"} Sep 10 12:07:25.182210 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.182094 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" event={"ID":"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab","Type":"ContainerStarted","Data":"6d6ce1ce767623d8906409450405a818c73976a50c3211db04a4fb69d0bb24ae"} Sep 10 12:07:25.182210 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.182126 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" event={"ID":"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab","Type":"ContainerStarted","Data":"369fd077b50605024ac51a39c3426f2edc01f275a3bb14846cdb9a5f89bf1eea"} Sep 10 12:07:25.550807 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.550321 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:25.552633 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.552612 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 12:07:25.564530 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.564504 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e290b1-c39f-4ab8-b6ed-9584e49b3ae4-metrics-certs\") pod \"network-metrics-daemon-s8hwl\" (UID: \"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4\") " pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:25.638040 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.638006 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:07:25.651701 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.651257 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:07:25.653670 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.653646 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 12:07:25.669436 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.669399 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faf4ab54-b2cc-42bf-a906-e23f8d7949d9-original-pull-secret\") pod \"global-pull-secret-syncer-7wxnx\" (UID: \"faf4ab54-b2cc-42bf-a906-e23f8d7949d9\") " pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:07:25.682419 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.682384 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv"] Sep 10 12:07:25.686983 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.686955 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.690672 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.689944 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e1rqnd5277hj7\"" Sep 10 12:07:25.690672 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.690142 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-j4vcg\"" Sep 10 12:07:25.690672 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.690287 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 10 12:07:25.690672 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.690480 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 10 12:07:25.690672 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.690501 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 10 12:07:25.690672 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.690511 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 10 12:07:25.692793 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.692766 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 10 12:07:25.707567 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.707471 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv"] Sep 10 12:07:25.712924 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.712898 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wxnx" Sep 10 12:07:25.751845 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.751809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:25.756025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.755927 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 12:07:25.765764 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.765733 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 12:07:25.776326 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.776263 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb2ck\" (UniqueName: \"kubernetes.io/projected/3323f642-5184-411e-8db6-d639e9259107-kube-api-access-gb2ck\") pod \"network-check-target-9wlph\" (UID: \"3323f642-5184-411e-8db6-d639e9259107\") " pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:25.807146 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.806891 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-llnv7\"" Sep 10 12:07:25.815167 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.814914 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8hwl" Sep 10 12:07:25.852977 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.852930 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.853147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.852997 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.853147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.853049 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.853147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.853079 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0512f0cb-b28b-4543-a8cc-7470058ebe9d-metrics-client-ca\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.853147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.853099 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-grpc-tls\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.853147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.853125 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-tls\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.853456 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.853148 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.853456 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.853176 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq78r\" (UniqueName: \"kubernetes.io/projected/0512f0cb-b28b-4543-a8cc-7470058ebe9d-kube-api-access-nq78r\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.953629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.953548 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.953629 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.953620 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.954119 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.953647 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0512f0cb-b28b-4543-a8cc-7470058ebe9d-metrics-client-ca\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.954119 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.953674 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-grpc-tls\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.954119 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.953707 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-tls\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.954119 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.953745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.954119 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.953777 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq78r\" (UniqueName: \"kubernetes.io/projected/0512f0cb-b28b-4543-a8cc-7470058ebe9d-kube-api-access-nq78r\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.954119 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.953833 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.955425 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.955401 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0512f0cb-b28b-4543-a8cc-7470058ebe9d-metrics-client-ca\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.957443 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.957414 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.957559 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.957518 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-tls\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.958160 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.958129 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.958290 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.958266 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.959386 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.959339 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-grpc-tls\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.959638 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.959617 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0512f0cb-b28b-4543-a8cc-7470058ebe9d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.966663 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:25.966626 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq78r\" (UniqueName: \"kubernetes.io/projected/0512f0cb-b28b-4543-a8cc-7470058ebe9d-kube-api-access-nq78r\") pod \"thanos-querier-5d48ffc5b4-tbtpv\" (UID: \"0512f0cb-b28b-4543-a8cc-7470058ebe9d\") " pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:25.998453 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:25.998420 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf17e9fd_0f7d_4dee_8ca9_7555880beab7.slice/crio-68dae311357e3bca72378fca98ca431ce05ca94b2af3127b2c768392a9577813 WatchSource:0}: Error finding container 68dae311357e3bca72378fca98ca431ce05ca94b2af3127b2c768392a9577813: Status 404 returned error can't find the container with id 68dae311357e3bca72378fca98ca431ce05ca94b2af3127b2c768392a9577813 Sep 10 12:07:26.003740 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.003222 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:26.010126 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.010101 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vr5lb\"" Sep 10 12:07:26.019045 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.019014 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:26.187798 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.187764 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerStarted","Data":"68dae311357e3bca72378fca98ca431ce05ca94b2af3127b2c768392a9577813"} Sep 10 12:07:26.190019 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.189988 2583 generic.go:356] "Generic (PLEG): container finished" podID="3510d8b5-99e5-4726-a1f3-177d939fe640" containerID="7826dfcb6786ac4e4dd1f64b3b5e329ab9b11b45357ae771837e5993c6e97ea3" exitCode=0 Sep 10 12:07:26.190162 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.190063 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcjkz" event={"ID":"3510d8b5-99e5-4726-a1f3-177d939fe640","Type":"ContainerDied","Data":"7826dfcb6786ac4e4dd1f64b3b5e329ab9b11b45357ae771837e5993c6e97ea3"} Sep 10 12:07:26.641555 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.641428 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s8hwl"] Sep 10 12:07:26.642802 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.642774 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7wxnx"] Sep 10 12:07:26.646384 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:26.646339 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e290b1_c39f_4ab8_b6ed_9584e49b3ae4.slice/crio-914b23f8695a7fad16f4e24d28735a79dced683e34d9c4558e053f59782b6f9c WatchSource:0}: Error finding container 914b23f8695a7fad16f4e24d28735a79dced683e34d9c4558e053f59782b6f9c: Status 404 returned error can't find the container with id 914b23f8695a7fad16f4e24d28735a79dced683e34d9c4558e053f59782b6f9c Sep 10 12:07:26.654720 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:26.654690 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf4ab54_b2cc_42bf_a906_e23f8d7949d9.slice/crio-c83ffe467b268095e0623c003c51f49283d0f2b41d79ab67110dcb4b936230ba WatchSource:0}: Error finding container c83ffe467b268095e0623c003c51f49283d0f2b41d79ab67110dcb4b936230ba: Status 404 returned error can't find the container with id c83ffe467b268095e0623c003c51f49283d0f2b41d79ab67110dcb4b936230ba Sep 10 12:07:26.694255 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.694218 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9wlph"] Sep 10 12:07:26.697902 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:26.697870 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3323f642_5184_411e_8db6_d639e9259107.slice/crio-9523bf4a500589dfdeb8b35568a1623644c0116ea9ba3e84c0256309d3911c17 WatchSource:0}: Error finding container 9523bf4a500589dfdeb8b35568a1623644c0116ea9ba3e84c0256309d3911c17: Status 404 returned error can't find the container with id 9523bf4a500589dfdeb8b35568a1623644c0116ea9ba3e84c0256309d3911c17 Sep 10 12:07:26.707693 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:26.707667 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv"] Sep 10 12:07:26.710134 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:26.710102 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0512f0cb_b28b_4543_a8cc_7470058ebe9d.slice/crio-21513a141dece84292ccac7a9479d4465be9244d571ec99762c9f951c0201924 WatchSource:0}: Error finding container 21513a141dece84292ccac7a9479d4465be9244d571ec99762c9f951c0201924: Status 404 returned error can't find the container with id 21513a141dece84292ccac7a9479d4465be9244d571ec99762c9f951c0201924 Sep 10 12:07:27.146823 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.146789 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-55t8l" Sep 10 12:07:27.197896 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.197852 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9wlph" event={"ID":"3323f642-5184-411e-8db6-d639e9259107","Type":"ContainerStarted","Data":"9523bf4a500589dfdeb8b35568a1623644c0116ea9ba3e84c0256309d3911c17"} Sep 10 12:07:27.200793 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.200701 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" event={"ID":"0512f0cb-b28b-4543-a8cc-7470058ebe9d","Type":"ContainerStarted","Data":"21513a141dece84292ccac7a9479d4465be9244d571ec99762c9f951c0201924"} Sep 10 12:07:27.202339 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.202276 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7wxnx" event={"ID":"faf4ab54-b2cc-42bf-a906-e23f8d7949d9","Type":"ContainerStarted","Data":"c83ffe467b268095e0623c003c51f49283d0f2b41d79ab67110dcb4b936230ba"} Sep 10 12:07:27.206809 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.206771 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcjkz" event={"ID":"3510d8b5-99e5-4726-a1f3-177d939fe640","Type":"ContainerStarted","Data":"b41f0f37b9cad6f0bb5b88406dbaed47d31c28f3e4bb3e3789ba824dec94318b"} Sep 10 12:07:27.206940 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.206839 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcjkz" event={"ID":"3510d8b5-99e5-4726-a1f3-177d939fe640","Type":"ContainerStarted","Data":"74ea1c93ca555195c7f37dead66fb7e686f389debe6b0aa27ce4ccea75be539b"} Sep 10 12:07:27.212616 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.212584 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" event={"ID":"2d5d9d44-6619-4706-9689-b66c12635aa6","Type":"ContainerStarted","Data":"49eb126fd4d61bcaab820d50878915ca9e0397e3cd2882f72eab6be0d6f3c932"} Sep 10 12:07:27.212758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.212649 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" event={"ID":"2d5d9d44-6619-4706-9689-b66c12635aa6","Type":"ContainerStarted","Data":"bccb126b611b4f8a3b355fdbea1f01a87af03c10ea4cf68e3d5424c5cc55a6e1"} Sep 10 12:07:27.212758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.212720 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" event={"ID":"2d5d9d44-6619-4706-9689-b66c12635aa6","Type":"ContainerStarted","Data":"959ea4714270ae794eb7a46151d7669683847a5d82c77f8d35553018c895854a"} Sep 10 12:07:27.215268 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.215236 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s8hwl" event={"ID":"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4","Type":"ContainerStarted","Data":"914b23f8695a7fad16f4e24d28735a79dced683e34d9c4558e053f59782b6f9c"} Sep 10 12:07:27.219089 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.219024 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" event={"ID":"152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab","Type":"ContainerStarted","Data":"0d3751de49a408ad548aaa9480a7ee1093c66268b55cb4b5243650e79e54530e"} Sep 10 12:07:27.238151 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.237666 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qcjkz" podStartSLOduration=2.805779729 podStartE2EDuration="4.237642573s" totalImagesPullingTime="1.347264896s" totalInitContainerRuntime="84.597948ms" isStatefulPod=true podCreationTimestamp="2026-09-10 12:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:27.236126734 +0000 UTC m=+67.813450344" watchObservedRunningTime="2026-09-10 12:07:27.237642573 +0000 UTC m=+67.814966182" Sep 10 12:07:27.264622 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.264555 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fvs6v" podStartSLOduration=2.3824970309999998 podStartE2EDuration="4.264535969s" totalImagesPullingTime="1.882038938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:27.264390188 +0000 UTC m=+67.841713796" watchObservedRunningTime="2026-09-10 12:07:27.264535969 +0000 UTC m=+67.841859579" Sep 10 12:07:27.291718 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.291613 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t5j77" podStartSLOduration=2.274408018 podStartE2EDuration="4.291568772s" totalImagesPullingTime="2.017160754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:27.290534591 +0000 UTC m=+67.867858203" watchObservedRunningTime="2026-09-10 12:07:27.291568772 +0000 UTC m=+67.868892383" Sep 10 12:07:27.653081 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.651718 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59c756ddc8-r65tz"] Sep 10 12:07:27.656132 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.655222 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.658814 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.658788 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h7hf8\"" Sep 10 12:07:27.659746 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.659725 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 12:07:27.660425 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.660406 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 12:07:27.660650 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.660633 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 12:07:27.660847 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.660832 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 12:07:27.661285 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.661265 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 12:07:27.674858 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.674821 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c756ddc8-r65tz"] Sep 10 12:07:27.774617 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.774581 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-oauth-serving-cert\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.774798 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.774643 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-config\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.774798 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.774678 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-serving-cert\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.774798 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.774705 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr42h\" (UniqueName: \"kubernetes.io/projected/afc67bd2-01b3-4c52-aec2-24943a7b1302-kube-api-access-jr42h\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.774798 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.774739 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-service-ca\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.774798 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.774767 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-oauth-config\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.876768 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.875760 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-serving-cert\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.876768 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.875813 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr42h\" (UniqueName: \"kubernetes.io/projected/afc67bd2-01b3-4c52-aec2-24943a7b1302-kube-api-access-jr42h\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.876768 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.875857 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-service-ca\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.876768 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.875957 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-oauth-config\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.876768 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.876080 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-oauth-serving-cert\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.876768 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.876119 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-config\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.876768 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.876721 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-service-ca\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.878279 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.877793 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-config\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.878279 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.878237 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-oauth-serving-cert\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.888386 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.884465 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-serving-cert\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.888386 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.886691 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr42h\" (UniqueName: \"kubernetes.io/projected/afc67bd2-01b3-4c52-aec2-24943a7b1302-kube-api-access-jr42h\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.888694 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.888670 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-oauth-config\") pod \"console-59c756ddc8-r65tz\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:27.969570 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:27.969494 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:28.183300 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.183259 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-786964d67d-s7btn"] Sep 10 12:07:28.186838 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.186772 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.190437 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.190381 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 10 12:07:28.190560 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.190483 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vklkm\"" Sep 10 12:07:28.190734 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.190719 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 10 12:07:28.190994 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.190977 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2p07iksi41dt2\"" Sep 10 12:07:28.191328 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.191113 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 10 12:07:28.191328 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.191245 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 12:07:28.210959 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.210559 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-786964d67d-s7btn"] Sep 10 12:07:28.280591 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.279506 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ca347c44-8285-4f20-aaeb-663f8b27b0ee-secret-metrics-server-tls\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.280591 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.279555 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ca347c44-8285-4f20-aaeb-663f8b27b0ee-metrics-server-audit-profiles\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.280591 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.279576 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ca347c44-8285-4f20-aaeb-663f8b27b0ee-client-ca-bundle\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.280591 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.279592 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b67wh\" (UniqueName: \"kubernetes.io/projected/ca347c44-8285-4f20-aaeb-663f8b27b0ee-kube-api-access-b67wh\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.280591 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.279608 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ca347c44-8285-4f20-aaeb-663f8b27b0ee-secret-metrics-server-client-certs\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.280591 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.279675 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca347c44-8285-4f20-aaeb-663f8b27b0ee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.280591 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.279704 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ca347c44-8285-4f20-aaeb-663f8b27b0ee-audit-log\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.342064 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.342025 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b"] Sep 10 12:07:28.345968 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.345939 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:28.350063 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.350036 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 12:07:28.350585 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.350299 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5nngx\"" Sep 10 12:07:28.357745 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.357702 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b"] Sep 10 12:07:28.380771 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.380738 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca347c44-8285-4f20-aaeb-663f8b27b0ee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.380920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.380786 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ca347c44-8285-4f20-aaeb-663f8b27b0ee-audit-log\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.380920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.380866 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ca347c44-8285-4f20-aaeb-663f8b27b0ee-secret-metrics-server-tls\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.380920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.380887 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ca347c44-8285-4f20-aaeb-663f8b27b0ee-metrics-server-audit-profiles\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.380920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.380903 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ca347c44-8285-4f20-aaeb-663f8b27b0ee-client-ca-bundle\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.380920 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.380918 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b67wh\" (UniqueName: \"kubernetes.io/projected/ca347c44-8285-4f20-aaeb-663f8b27b0ee-kube-api-access-b67wh\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.381104 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.380950 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ca347c44-8285-4f20-aaeb-663f8b27b0ee-secret-metrics-server-client-certs\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.382872 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.382196 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ca347c44-8285-4f20-aaeb-663f8b27b0ee-audit-log\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.382872 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.382837 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca347c44-8285-4f20-aaeb-663f8b27b0ee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.383809 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.383731 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ca347c44-8285-4f20-aaeb-663f8b27b0ee-metrics-server-audit-profiles\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.385414 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.385346 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ca347c44-8285-4f20-aaeb-663f8b27b0ee-secret-metrics-server-client-certs\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.386465 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.386421 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ca347c44-8285-4f20-aaeb-663f8b27b0ee-secret-metrics-server-tls\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.386925 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.386892 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ca347c44-8285-4f20-aaeb-663f8b27b0ee-client-ca-bundle\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.395641 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.395590 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b67wh\" (UniqueName: \"kubernetes.io/projected/ca347c44-8285-4f20-aaeb-663f8b27b0ee-kube-api-access-b67wh\") pod \"metrics-server-786964d67d-s7btn\" (UID: \"ca347c44-8285-4f20-aaeb-663f8b27b0ee\") " pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.485224 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.484894 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vplc4\" (UniqueName: \"kubernetes.io/projected/b987202e-934b-4ad0-be06-489809b415ae-kube-api-access-vplc4\") pod \"monitoring-plugin-85bbbbbf4-xwr7b\" (UID: \"b987202e-934b-4ad0-be06-489809b415ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:28.485224 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.484977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b987202e-934b-4ad0-be06-489809b415ae-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-xwr7b\" (UID: \"b987202e-934b-4ad0-be06-489809b415ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:28.502387 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.502233 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:28.586557 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.585846 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vplc4\" (UniqueName: \"kubernetes.io/projected/b987202e-934b-4ad0-be06-489809b415ae-kube-api-access-vplc4\") pod \"monitoring-plugin-85bbbbbf4-xwr7b\" (UID: \"b987202e-934b-4ad0-be06-489809b415ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:28.586557 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.585910 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b987202e-934b-4ad0-be06-489809b415ae-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-xwr7b\" (UID: \"b987202e-934b-4ad0-be06-489809b415ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:28.586557 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:28.586031 2583 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 10 12:07:28.586557 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:07:28.586090 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b987202e-934b-4ad0-be06-489809b415ae-monitoring-plugin-cert podName:b987202e-934b-4ad0-be06-489809b415ae nodeName:}" failed. No retries permitted until 2026-09-10 12:07:29.086071265 +0000 UTC m=+69.663394857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b987202e-934b-4ad0-be06-489809b415ae-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-xwr7b" (UID: "b987202e-934b-4ad0-be06-489809b415ae") : secret "monitoring-plugin-cert" not found Sep 10 12:07:28.603838 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.603777 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vplc4\" (UniqueName: \"kubernetes.io/projected/b987202e-934b-4ad0-be06-489809b415ae-kube-api-access-vplc4\") pod \"monitoring-plugin-85bbbbbf4-xwr7b\" (UID: \"b987202e-934b-4ad0-be06-489809b415ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:28.749532 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.749480 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c756ddc8-r65tz"] Sep 10 12:07:28.751917 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:28.751875 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc67bd2_01b3_4c52_aec2_24943a7b1302.slice/crio-b15fe8c10419bc771edee736df00d87bdf31d3605fbcdef4f62b8be990636e74 WatchSource:0}: Error finding container b15fe8c10419bc771edee736df00d87bdf31d3605fbcdef4f62b8be990636e74: Status 404 returned error can't find the container with id b15fe8c10419bc771edee736df00d87bdf31d3605fbcdef4f62b8be990636e74 Sep 10 12:07:28.811344 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:28.811304 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-786964d67d-s7btn"] Sep 10 12:07:28.815887 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:28.815853 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca347c44_8285_4f20_aaeb_663f8b27b0ee.slice/crio-f860ccb58d5c2be422f5bf8a1e6835be89f373191c39a844451bbb5c21372063 WatchSource:0}: Error finding container f860ccb58d5c2be422f5bf8a1e6835be89f373191c39a844451bbb5c21372063: Status 404 returned error can't find the container with id f860ccb58d5c2be422f5bf8a1e6835be89f373191c39a844451bbb5c21372063 Sep 10 12:07:29.090546 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.090339 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b987202e-934b-4ad0-be06-489809b415ae-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-xwr7b\" (UID: \"b987202e-934b-4ad0-be06-489809b415ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:29.094471 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.094413 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b987202e-934b-4ad0-be06-489809b415ae-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-xwr7b\" (UID: \"b987202e-934b-4ad0-be06-489809b415ae\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:29.242884 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.242803 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-786964d67d-s7btn" event={"ID":"ca347c44-8285-4f20-aaeb-663f8b27b0ee","Type":"ContainerStarted","Data":"f860ccb58d5c2be422f5bf8a1e6835be89f373191c39a844451bbb5c21372063"} Sep 10 12:07:29.245610 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.245574 2583 generic.go:356] "Generic (PLEG): container finished" podID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerID="c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67" exitCode=0 Sep 10 12:07:29.245739 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.245658 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerDied","Data":"c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67"} Sep 10 12:07:29.248523 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.248498 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c756ddc8-r65tz" event={"ID":"afc67bd2-01b3-4c52-aec2-24943a7b1302","Type":"ContainerStarted","Data":"b15fe8c10419bc771edee736df00d87bdf31d3605fbcdef4f62b8be990636e74"} Sep 10 12:07:29.253961 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.253921 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s8hwl" event={"ID":"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4","Type":"ContainerStarted","Data":"d0b1378c5dfc7c89cd702c9f9e694b867cba3d9fc2250dc0a2f15e23365f55f9"} Sep 10 12:07:29.254125 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.253976 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s8hwl" event={"ID":"35e290b1-c39f-4ab8-b6ed-9584e49b3ae4","Type":"ContainerStarted","Data":"767412ce74a580c6ac776d8a98907235c8937bf3a9e5cb7ebaa1f2b597a63312"} Sep 10 12:07:29.262141 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.262116 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:29.304382 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.302233 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s8hwl" podStartSLOduration=68.508025776 podStartE2EDuration="1m10.302210617s" totalImagesPullingTime="1.794184841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:29.300924921 +0000 UTC m=+69.878248531" watchObservedRunningTime="2026-09-10 12:07:29.302210617 +0000 UTC m=+69.879534240" Sep 10 12:07:29.569586 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:29.569488 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b"] Sep 10 12:07:30.061163 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.058914 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:07:30.064605 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.064573 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.074645 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.074149 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 12:07:30.075487 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.075166 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 12:07:30.082528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.081014 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 12:07:30.082528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.081680 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mfz87\"" Sep 10 12:07:30.082528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.081951 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 12:07:30.082528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.082036 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 12:07:30.082528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.082090 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fo0762qlveoan\"" Sep 10 12:07:30.082528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.082322 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 12:07:30.082528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.082341 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 12:07:30.082528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.082424 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 12:07:30.083088 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.082610 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 12:07:30.083144 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.083090 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 12:07:30.083506 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.083292 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 12:07:30.083607 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.083574 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 12:07:30.083607 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.083598 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 12:07:30.091140 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.091117 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 12:07:30.093987 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.093964 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:07:30.202742 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202700 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.202929 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202757 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.202929 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202794 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-config\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.202929 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202820 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.202929 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202860 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.202929 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202887 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.202929 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202915 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202938 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rl7z\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-kube-api-access-4rl7z\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.202992 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203025 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203062 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-config-out\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203086 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203127 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203150 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203177 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-web-config\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203211 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203243 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203816 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203294 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.203816 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.203319 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.266980 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.266946 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" event={"ID":"b987202e-934b-4ad0-be06-489809b415ae","Type":"ContainerStarted","Data":"a86018ae41467886d6387eb2a66626c613aa41aaa17af545377b5839509bf026"} Sep 10 12:07:30.304459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.304123 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.304459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.304181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.304459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.304221 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-config\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.304459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.304248 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.304459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.304295 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.304459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.304323 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.304459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.304363 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.304459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.304386 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rl7z\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-kube-api-access-4rl7z\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.304459 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.304409 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306000 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.305967 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306128 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306022 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306210 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306190 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306261 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306244 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306314 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306289 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-config-out\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306392 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306314 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306392 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306373 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306489 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306400 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306489 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306442 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-web-config\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306604 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306496 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306604 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306604 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.306739 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.306601 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.307997 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.307724 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.308118 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.308057 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.309095 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.308800 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.310943 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.310867 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.311443 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.311386 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.311955 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.311927 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.312577 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.312550 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.316525 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.316500 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.316904 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.316882 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.316999 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.316908 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-config\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.316999 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.316964 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.317133 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.317117 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.318634 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.318593 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-config-out\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.318854 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.318827 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.319480 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.319442 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-web-config\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.319989 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.319963 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.320076 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.320059 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.323102 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.323059 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rl7z\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-kube-api-access-4rl7z\") pod \"prometheus-k8s-0\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:30.390989 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:30.390943 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:07:44.189045 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:44.189021 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:07:44.277236 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:07:44.277205 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eef4e24_cf25_4965_81d5_6ab964cadd16.slice/crio-2de219bf12a0581dbe8fd70e04a570a7b3c336f6aa574eff42d6002f328c3304 WatchSource:0}: Error finding container 2de219bf12a0581dbe8fd70e04a570a7b3c336f6aa574eff42d6002f328c3304: Status 404 returned error can't find the container with id 2de219bf12a0581dbe8fd70e04a570a7b3c336f6aa574eff42d6002f328c3304 Sep 10 12:07:44.330117 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:44.330089 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerStarted","Data":"2de219bf12a0581dbe8fd70e04a570a7b3c336f6aa574eff42d6002f328c3304"} Sep 10 12:07:45.337828 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.337778 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" event={"ID":"b987202e-934b-4ad0-be06-489809b415ae","Type":"ContainerStarted","Data":"ce07d34c962ce819582ba422ee1887e6ca0e20e6365eb717a573baea7b8789c1"} Sep 10 12:07:45.338283 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.337886 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:45.339914 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.339885 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-786964d67d-s7btn" event={"ID":"ca347c44-8285-4f20-aaeb-663f8b27b0ee","Type":"ContainerStarted","Data":"a534500dcb0754cc8940e9fa5bdabb076e8a4f6f7020a94add7ace7cb46e0234"} Sep 10 12:07:45.342278 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.342250 2583 generic.go:356] "Generic (PLEG): container finished" podID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerID="89694fcc4e467b972622cc171d37382f26c93128c4940645f59248e3f271f1e3" exitCode=0 Sep 10 12:07:45.342455 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.342414 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerDied","Data":"89694fcc4e467b972622cc171d37382f26c93128c4940645f59248e3f271f1e3"} Sep 10 12:07:45.344944 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.344923 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" Sep 10 12:07:45.345225 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.345203 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-6ftmg" event={"ID":"cbbf08dd-74e3-476f-89f1-a883cb9b5054","Type":"ContainerStarted","Data":"66f8334208b7f912d337a78732ef0cbe755c9165b98b568da58022dbfd12986f"} Sep 10 12:07:45.345420 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.345400 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-6ftmg" Sep 10 12:07:45.349031 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.349004 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerStarted","Data":"2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7"} Sep 10 12:07:45.349120 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.349077 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerStarted","Data":"2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df"} Sep 10 12:07:45.349120 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.349115 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerStarted","Data":"c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323"} Sep 10 12:07:45.349202 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.349141 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerStarted","Data":"35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0"} Sep 10 12:07:45.349202 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.349165 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerStarted","Data":"3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2"} Sep 10 12:07:45.350878 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.350853 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9wlph" event={"ID":"3323f642-5184-411e-8db6-d639e9259107","Type":"ContainerStarted","Data":"1edb9e52b780c641eb03da6d4da7d97791554fe3fc0c1c409a90b3be5b73032c"} Sep 10 12:07:45.351102 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.350993 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:07:45.353326 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.353305 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" event={"ID":"0512f0cb-b28b-4543-a8cc-7470058ebe9d","Type":"ContainerStarted","Data":"d2f9351977f02e0dae03ef60ca44bdb7104b1bda2deeb21a3d123e48852241a2"} Sep 10 12:07:45.353431 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.353389 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" event={"ID":"0512f0cb-b28b-4543-a8cc-7470058ebe9d","Type":"ContainerStarted","Data":"3fe5634686d75e121e02009b86a981a0cea4bf227431ae48ac55aac2a5f4ecbe"} Sep 10 12:07:45.353494 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.353433 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" event={"ID":"0512f0cb-b28b-4543-a8cc-7470058ebe9d","Type":"ContainerStarted","Data":"09c34703eb64ab3522485d7ae1ae97a69945f6eef7a94f26420645dfb37dda73"} Sep 10 12:07:45.355134 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.355111 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7wxnx" event={"ID":"faf4ab54-b2cc-42bf-a906-e23f8d7949d9","Type":"ContainerStarted","Data":"5ff45832b579fab3d16ef0848546f407530b8479cac92f4aa7d19d24adb484e5"} Sep 10 12:07:45.357073 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.357048 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c756ddc8-r65tz" event={"ID":"afc67bd2-01b3-4c52-aec2-24943a7b1302","Type":"ContainerStarted","Data":"9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0"} Sep 10 12:07:45.360763 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.360723 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xwr7b" podStartSLOduration=2.645173284 podStartE2EDuration="17.360710085s" totalImagesPullingTime="14.715536801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:45.359603941 +0000 UTC m=+85.936927543" watchObservedRunningTime="2026-09-10 12:07:45.360710085 +0000 UTC m=+85.938033697" Sep 10 12:07:45.361445 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.361415 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-6ftmg" Sep 10 12:07:45.401501 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.401439 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9wlph" podStartSLOduration=67.803157241 podStartE2EDuration="1m25.401416338s" totalImagesPullingTime="17.598259097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:45.399788106 +0000 UTC m=+85.977111790" watchObservedRunningTime="2026-09-10 12:07:45.401416338 +0000 UTC m=+85.978739948" Sep 10 12:07:45.402507 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.402462 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7wxnx" podStartSLOduration=67.757586721 podStartE2EDuration="1m25.402453293s" totalImagesPullingTime="17.644866572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:45.380656111 +0000 UTC m=+85.957979720" watchObservedRunningTime="2026-09-10 12:07:45.402453293 +0000 UTC m=+85.979776896" Sep 10 12:07:45.432093 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.432032 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-786964d67d-s7btn" podStartSLOduration=1.952150723 podStartE2EDuration="17.432008433s" totalImagesPullingTime="15.47985771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:45.430279493 +0000 UTC m=+86.007603103" watchObservedRunningTime="2026-09-10 12:07:45.432008433 +0000 UTC m=+86.009332042" Sep 10 12:07:45.463785 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.463724 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59c756ddc8-r65tz" podStartSLOduration=2.915525532 podStartE2EDuration="18.463707048s" totalImagesPullingTime="15.548181516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:45.462302729 +0000 UTC m=+86.039626340" watchObservedRunningTime="2026-09-10 12:07:45.463707048 +0000 UTC m=+86.041030660" Sep 10 12:07:45.570632 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:45.570567 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-6ftmg" podStartSLOduration=2.52010454 podStartE2EDuration="28.570543455s" totalImagesPullingTime="26.050438915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:45.566518279 +0000 UTC m=+86.143841891" watchObservedRunningTime="2026-09-10 12:07:45.570543455 +0000 UTC m=+86.147867066" Sep 10 12:07:46.389622 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:46.389584 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c756ddc8-r65tz"] Sep 10 12:07:47.370615 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:47.370575 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerStarted","Data":"f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553"} Sep 10 12:07:47.375393 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:47.375324 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" event={"ID":"0512f0cb-b28b-4543-a8cc-7470058ebe9d","Type":"ContainerStarted","Data":"f4abfd083492ca94cc330a2bcc3e3e6581ad966652e3671a0068b1e7dae3fc2e"} Sep 10 12:07:47.375561 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:47.375443 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" event={"ID":"0512f0cb-b28b-4543-a8cc-7470058ebe9d","Type":"ContainerStarted","Data":"f6058132cd53106c1825a72b0de3ca5f7e2e6a4add10abdc1e80bfa07b54a495"} Sep 10 12:07:47.375561 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:47.375488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" event={"ID":"0512f0cb-b28b-4543-a8cc-7470058ebe9d","Type":"ContainerStarted","Data":"cd24f1426cdf1b7e9d547b30957c161fca7e62d8bff94aae3acc94bec3b40822"} Sep 10 12:07:47.416297 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:47.416232 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.982670391 podStartE2EDuration="23.416213578s" totalImagesPullingTime="19.408642433s" totalInitContainerRuntime="24.900754ms" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:47.415134799 +0000 UTC m=+87.992458416" watchObservedRunningTime="2026-09-10 12:07:47.416213578 +0000 UTC m=+87.993537186" Sep 10 12:07:47.453718 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:47.451775 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" podStartSLOduration=2.734592931 podStartE2EDuration="22.451752603s" totalImagesPullingTime="19.717159672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:47.450089281 +0000 UTC m=+88.027412927" watchObservedRunningTime="2026-09-10 12:07:47.451752603 +0000 UTC m=+88.029076214" Sep 10 12:07:47.970108 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:47.970068 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:07:48.382022 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:48.381984 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:48.502814 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:48.502775 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:48.503236 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:48.502844 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:07:49.400697 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:49.400667 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d48ffc5b4-tbtpv" Sep 10 12:07:51.400409 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:51.400371 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerStarted","Data":"28b84d07c993649bfafc16ab8d92d7fe544875a32e281cfc915145bd9b0bd14f"} Sep 10 12:07:51.400882 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:51.400460 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerStarted","Data":"6f6889d03031fa69f1d48d9c389c4d33329f1d6cb47c9addab5c2efa9071e391"} Sep 10 12:07:51.400882 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:51.400511 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerStarted","Data":"ea574aab11b0c1c86a7625f75d4f2ade642636d2d08be4fda23487496d6f0267"} Sep 10 12:07:51.400882 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:51.400550 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerStarted","Data":"87c313d308795a4b0a569d4adcdd3b87d52a680119913742c6e7f35679d14b27"} Sep 10 12:07:51.400882 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:51.400592 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerStarted","Data":"2dea52b4e5ca622b3d7f8d8c0f656d38996737739c265134ffa661cbfdb4abd8"} Sep 10 12:07:51.400882 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:51.400636 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerStarted","Data":"454bd4bb6d9e670a264b5d016d435384238cf2ea0fd06bced9a989e9d8f3d20e"} Sep 10 12:07:51.434423 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:51.434294 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=16.394335637 podStartE2EDuration="21.434265715s" totalImagesPullingTime="4.979371268s" totalInitContainerRuntime="60.55881ms" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:51.432211811 +0000 UTC m=+92.009535420" watchObservedRunningTime="2026-09-10 12:07:51.434265715 +0000 UTC m=+92.011589384" Sep 10 12:07:55.391164 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:07:55.391119 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:08.508845 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:08.508817 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:08:08.512872 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:08.512847 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-786964d67d-s7btn" Sep 10 12:08:12.399288 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.399244 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59c756ddc8-r65tz" podUID="afc67bd2-01b3-4c52-aec2-24943a7b1302" containerName="console" containerID="cri-o://9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0" gracePeriod=15 Sep 10 12:08:12.648474 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.648439 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:08:12.707110 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.707032 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-config\") pod \"afc67bd2-01b3-4c52-aec2-24943a7b1302\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " Sep 10 12:08:12.707110 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.707081 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-serving-cert\") pod \"afc67bd2-01b3-4c52-aec2-24943a7b1302\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " Sep 10 12:08:12.707110 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.707105 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-oauth-serving-cert\") pod \"afc67bd2-01b3-4c52-aec2-24943a7b1302\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " Sep 10 12:08:12.707400 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.707175 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-service-ca\") pod \"afc67bd2-01b3-4c52-aec2-24943a7b1302\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " Sep 10 12:08:12.707400 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.707192 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-oauth-config\") pod \"afc67bd2-01b3-4c52-aec2-24943a7b1302\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " Sep 10 12:08:12.707400 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.707233 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr42h\" (UniqueName: \"kubernetes.io/projected/afc67bd2-01b3-4c52-aec2-24943a7b1302-kube-api-access-jr42h\") pod \"afc67bd2-01b3-4c52-aec2-24943a7b1302\" (UID: \"afc67bd2-01b3-4c52-aec2-24943a7b1302\") " Sep 10 12:08:12.707562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.707486 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-config" (OuterVolumeSpecName: "console-config") pod "afc67bd2-01b3-4c52-aec2-24943a7b1302" (UID: "afc67bd2-01b3-4c52-aec2-24943a7b1302"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:12.707562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.707527 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-service-ca" (OuterVolumeSpecName: "service-ca") pod "afc67bd2-01b3-4c52-aec2-24943a7b1302" (UID: "afc67bd2-01b3-4c52-aec2-24943a7b1302"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:12.707669 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.707600 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "afc67bd2-01b3-4c52-aec2-24943a7b1302" (UID: "afc67bd2-01b3-4c52-aec2-24943a7b1302"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:12.709783 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.709754 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "afc67bd2-01b3-4c52-aec2-24943a7b1302" (UID: "afc67bd2-01b3-4c52-aec2-24943a7b1302"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:12.709919 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.709842 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afc67bd2-01b3-4c52-aec2-24943a7b1302-kube-api-access-jr42h" (OuterVolumeSpecName: "kube-api-access-jr42h") pod "afc67bd2-01b3-4c52-aec2-24943a7b1302" (UID: "afc67bd2-01b3-4c52-aec2-24943a7b1302"). InnerVolumeSpecName "kube-api-access-jr42h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:08:12.710239 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.710210 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "afc67bd2-01b3-4c52-aec2-24943a7b1302" (UID: "afc67bd2-01b3-4c52-aec2-24943a7b1302"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:12.808051 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.808015 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-service-ca\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:12.808051 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.808044 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-oauth-config\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:12.808051 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.808054 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr42h\" (UniqueName: \"kubernetes.io/projected/afc67bd2-01b3-4c52-aec2-24943a7b1302-kube-api-access-jr42h\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:12.808051 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.808064 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-config\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:12.808449 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.808073 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afc67bd2-01b3-4c52-aec2-24943a7b1302-console-serving-cert\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:12.808449 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:12.808081 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afc67bd2-01b3-4c52-aec2-24943a7b1302-oauth-serving-cert\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:13.501393 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.501336 2583 generic.go:356] "Generic (PLEG): container finished" podID="afc67bd2-01b3-4c52-aec2-24943a7b1302" containerID="9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0" exitCode=0 Sep 10 12:08:13.501917 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.501417 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c756ddc8-r65tz" Sep 10 12:08:13.501917 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.501427 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c756ddc8-r65tz" event={"ID":"afc67bd2-01b3-4c52-aec2-24943a7b1302","Type":"ContainerDied","Data":"9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0"} Sep 10 12:08:13.501917 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.501488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c756ddc8-r65tz" event={"ID":"afc67bd2-01b3-4c52-aec2-24943a7b1302","Type":"ContainerDied","Data":"b15fe8c10419bc771edee736df00d87bdf31d3605fbcdef4f62b8be990636e74"} Sep 10 12:08:13.501917 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.501515 2583 scope.go:117] "RemoveContainer" containerID="9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0" Sep 10 12:08:13.511221 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.511202 2583 scope.go:117] "RemoveContainer" containerID="9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0" Sep 10 12:08:13.511544 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:08:13.511512 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0\": container with ID starting with 9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0 not found: ID does not exist" containerID="9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0" Sep 10 12:08:13.511642 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.511541 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0"} err="failed to get container status \"9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0\": rpc error: code = NotFound desc = could not find container \"9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0\": container with ID starting with 9e41a32c7b46f29e2c2dc9db9943e7c18af65aba7d3de7ee3d767198e0a0b6d0 not found: ID does not exist" Sep 10 12:08:13.527539 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.527507 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c756ddc8-r65tz"] Sep 10 12:08:13.529263 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.529237 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59c756ddc8-r65tz"] Sep 10 12:08:13.901817 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:13.901780 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afc67bd2-01b3-4c52-aec2-24943a7b1302" path="/var/lib/kubelet/pods/afc67bd2-01b3-4c52-aec2-24943a7b1302/volumes" Sep 10 12:08:16.364308 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:16.364273 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9wlph" Sep 10 12:08:30.391711 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:30.391662 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:30.475615 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:30.475583 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:30.587242 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:30.587212 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:44.035241 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.035204 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:08:44.035787 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.035501 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="alertmanager" containerID="cri-o://3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2" gracePeriod=120 Sep 10 12:08:44.035787 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.035577 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy-metric" containerID="cri-o://2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7" gracePeriod=120 Sep 10 12:08:44.035787 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.035603 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy-web" containerID="cri-o://c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323" gracePeriod=120 Sep 10 12:08:44.035787 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.035644 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="prom-label-proxy" containerID="cri-o://f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553" gracePeriod=120 Sep 10 12:08:44.035787 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.035679 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy" containerID="cri-o://2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df" gracePeriod=120 Sep 10 12:08:44.035787 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.035695 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="config-reloader" containerID="cri-o://35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0" gracePeriod=120 Sep 10 12:08:44.628752 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.628715 2583 generic.go:356] "Generic (PLEG): container finished" podID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerID="f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553" exitCode=0 Sep 10 12:08:44.628752 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.628742 2583 generic.go:356] "Generic (PLEG): container finished" podID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerID="2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df" exitCode=0 Sep 10 12:08:44.628752 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.628749 2583 generic.go:356] "Generic (PLEG): container finished" podID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerID="35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0" exitCode=0 Sep 10 12:08:44.628752 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.628754 2583 generic.go:356] "Generic (PLEG): container finished" podID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerID="3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2" exitCode=0 Sep 10 12:08:44.629079 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.628790 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerDied","Data":"f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553"} Sep 10 12:08:44.629079 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.628902 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerDied","Data":"2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df"} Sep 10 12:08:44.629079 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.628942 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerDied","Data":"35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0"} Sep 10 12:08:44.629079 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:44.628983 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerDied","Data":"3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2"} Sep 10 12:08:45.274513 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.274485 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.392254 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.392171 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx4sj\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-kube-api-access-hx4sj\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.392254 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.392225 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-trusted-ca-bundle\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.392254 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.392255 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-web\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.392547 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.392289 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.392547 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.392316 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-out\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.392547 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.392333 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-cluster-tls-config\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.392547 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.392390 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.392758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.392675 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:45.392853 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.392829 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-main-tls\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.393267 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.393248 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-metrics-client-ca\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.393496 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.393479 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-volume\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.393754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.393716 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:45.393913 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.393896 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-main-db\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.394061 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.394046 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-tls-assets\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.394221 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.394198 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-web-config\") pod \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\" (UID: \"bf17e9fd-0f7d-4dee-8ca9-7555880beab7\") " Sep 10 12:08:45.394664 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.394106 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:08:45.395004 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.394983 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.395135 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.395121 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-metrics-client-ca\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.395235 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.395222 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-alertmanager-main-db\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.397147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.397116 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:45.397319 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.397296 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-out" (OuterVolumeSpecName: "config-out") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:08:45.397407 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.397370 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:45.397554 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.397520 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:45.398053 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.398008 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-volume" (OuterVolumeSpecName: "config-volume") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:45.398157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.398026 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-kube-api-access-hx4sj" (OuterVolumeSpecName: "kube-api-access-hx4sj") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "kube-api-access-hx4sj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:08:45.398157 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.398106 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:45.398273 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.398183 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:08:45.401473 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.401333 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:45.408875 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.408846 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-web-config" (OuterVolumeSpecName: "web-config") pod "bf17e9fd-0f7d-4dee-8ca9-7555880beab7" (UID: "bf17e9fd-0f7d-4dee-8ca9-7555880beab7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:45.495829 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495782 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx4sj\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-kube-api-access-hx4sj\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.495829 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495826 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.495829 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495837 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.496067 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495850 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-out\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.496067 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495858 2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-cluster-tls-config\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.496067 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495867 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.496067 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495876 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-secret-alertmanager-main-tls\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.496067 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495884 2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-config-volume\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.496067 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495891 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-tls-assets\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.496067 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.495899 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf17e9fd-0f7d-4dee-8ca9-7555880beab7-web-config\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:45.635436 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.635403 2583 generic.go:356] "Generic (PLEG): container finished" podID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerID="2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7" exitCode=0 Sep 10 12:08:45.635436 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.635429 2583 generic.go:356] "Generic (PLEG): container finished" podID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerID="c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323" exitCode=0 Sep 10 12:08:45.635645 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.635476 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.635645 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.635494 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerDied","Data":"2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7"} Sep 10 12:08:45.635645 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.635566 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerDied","Data":"c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323"} Sep 10 12:08:45.635645 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.635594 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf17e9fd-0f7d-4dee-8ca9-7555880beab7","Type":"ContainerDied","Data":"68dae311357e3bca72378fca98ca431ce05ca94b2af3127b2c768392a9577813"} Sep 10 12:08:45.635645 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.635628 2583 scope.go:117] "RemoveContainer" containerID="f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553" Sep 10 12:08:45.644406 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.644385 2583 scope.go:117] "RemoveContainer" containerID="2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7" Sep 10 12:08:45.651826 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.651806 2583 scope.go:117] "RemoveContainer" containerID="2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df" Sep 10 12:08:45.659799 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.659777 2583 scope.go:117] "RemoveContainer" containerID="c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323" Sep 10 12:08:45.660803 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.660772 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:08:45.667189 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.667170 2583 scope.go:117] "RemoveContainer" containerID="35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0" Sep 10 12:08:45.669965 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.669940 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:08:45.674979 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.674960 2583 scope.go:117] "RemoveContainer" containerID="3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2" Sep 10 12:08:45.682413 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.682395 2583 scope.go:117] "RemoveContainer" containerID="c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67" Sep 10 12:08:45.689236 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.689210 2583 scope.go:117] "RemoveContainer" containerID="f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553" Sep 10 12:08:45.689533 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:08:45.689515 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553\": container with ID starting with f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553 not found: ID does not exist" containerID="f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553" Sep 10 12:08:45.689600 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.689543 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553"} err="failed to get container status \"f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553\": rpc error: code = NotFound desc = could not find container \"f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553\": container with ID starting with f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553 not found: ID does not exist" Sep 10 12:08:45.689600 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.689562 2583 scope.go:117] "RemoveContainer" containerID="2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7" Sep 10 12:08:45.689860 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:08:45.689842 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7\": container with ID starting with 2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7 not found: ID does not exist" containerID="2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7" Sep 10 12:08:45.689908 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.689866 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7"} err="failed to get container status \"2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7\": rpc error: code = NotFound desc = could not find container \"2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7\": container with ID starting with 2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7 not found: ID does not exist" Sep 10 12:08:45.689908 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.689883 2583 scope.go:117] "RemoveContainer" containerID="2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df" Sep 10 12:08:45.690133 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:08:45.690112 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df\": container with ID starting with 2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df not found: ID does not exist" containerID="2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df" Sep 10 12:08:45.690220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.690139 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df"} err="failed to get container status \"2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df\": rpc error: code = NotFound desc = could not find container \"2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df\": container with ID starting with 2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df not found: ID does not exist" Sep 10 12:08:45.690220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.690158 2583 scope.go:117] "RemoveContainer" containerID="c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323" Sep 10 12:08:45.690435 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:08:45.690421 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323\": container with ID starting with c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323 not found: ID does not exist" containerID="c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323" Sep 10 12:08:45.690479 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.690437 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323"} err="failed to get container status \"c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323\": rpc error: code = NotFound desc = could not find container \"c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323\": container with ID starting with c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323 not found: ID does not exist" Sep 10 12:08:45.690479 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.690449 2583 scope.go:117] "RemoveContainer" containerID="35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0" Sep 10 12:08:45.690701 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:08:45.690686 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0\": container with ID starting with 35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0 not found: ID does not exist" containerID="35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0" Sep 10 12:08:45.690740 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.690705 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0"} err="failed to get container status \"35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0\": rpc error: code = NotFound desc = could not find container \"35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0\": container with ID starting with 35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0 not found: ID does not exist" Sep 10 12:08:45.690740 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.690717 2583 scope.go:117] "RemoveContainer" containerID="3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2" Sep 10 12:08:45.690929 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:08:45.690915 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2\": container with ID starting with 3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2 not found: ID does not exist" containerID="3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2" Sep 10 12:08:45.690968 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.690931 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2"} err="failed to get container status \"3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2\": rpc error: code = NotFound desc = could not find container \"3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2\": container with ID starting with 3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2 not found: ID does not exist" Sep 10 12:08:45.690968 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.690941 2583 scope.go:117] "RemoveContainer" containerID="c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67" Sep 10 12:08:45.691190 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:08:45.691174 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67\": container with ID starting with c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67 not found: ID does not exist" containerID="c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67" Sep 10 12:08:45.691227 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.691197 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67"} err="failed to get container status \"c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67\": rpc error: code = NotFound desc = could not find container \"c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67\": container with ID starting with c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67 not found: ID does not exist" Sep 10 12:08:45.691227 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.691214 2583 scope.go:117] "RemoveContainer" containerID="f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553" Sep 10 12:08:45.691449 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.691425 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553"} err="failed to get container status \"f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553\": rpc error: code = NotFound desc = could not find container \"f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553\": container with ID starting with f01d3b7c990dc2d945d13a78b31f6924de92b52f003dc6c6b0e7a94c669e0553 not found: ID does not exist" Sep 10 12:08:45.691449 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.691441 2583 scope.go:117] "RemoveContainer" containerID="2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7" Sep 10 12:08:45.691748 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.691669 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7"} err="failed to get container status \"2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7\": rpc error: code = NotFound desc = could not find container \"2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7\": container with ID starting with 2e1c97961280f3485595650352cf3ba23a6fb3f5bcbdc1e0526ddf81202ba6c7 not found: ID does not exist" Sep 10 12:08:45.691808 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.691752 2583 scope.go:117] "RemoveContainer" containerID="2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df" Sep 10 12:08:45.692017 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.691997 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df"} err="failed to get container status \"2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df\": rpc error: code = NotFound desc = could not find container \"2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df\": container with ID starting with 2650ec8434902f98f553e7a5d00b934b2ebf8acdd1f39e2bd8324d39ab1168df not found: ID does not exist" Sep 10 12:08:45.692058 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.692017 2583 scope.go:117] "RemoveContainer" containerID="c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323" Sep 10 12:08:45.692285 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.692261 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323"} err="failed to get container status \"c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323\": rpc error: code = NotFound desc = could not find container \"c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323\": container with ID starting with c442ddca6328f24c9b248a378b52eb921d89a568cb903f1ed8cb3067926a7323 not found: ID does not exist" Sep 10 12:08:45.692433 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.692318 2583 scope.go:117] "RemoveContainer" containerID="35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0" Sep 10 12:08:45.692810 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.692691 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0"} err="failed to get container status \"35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0\": rpc error: code = NotFound desc = could not find container \"35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0\": container with ID starting with 35e66e6c889e595df7f1e5375bde5f4d565162666bcc6c510c5985c6f056c1a0 not found: ID does not exist" Sep 10 12:08:45.692810 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.692718 2583 scope.go:117] "RemoveContainer" containerID="3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2" Sep 10 12:08:45.693143 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.693095 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2"} err="failed to get container status \"3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2\": rpc error: code = NotFound desc = could not find container \"3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2\": container with ID starting with 3220a5ad391d3171dc5e40f7cd756b9469831bd6d3a5dfb18b9043b9242226e2 not found: ID does not exist" Sep 10 12:08:45.693223 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.693146 2583 scope.go:117] "RemoveContainer" containerID="c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67" Sep 10 12:08:45.693477 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.693441 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67"} err="failed to get container status \"c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67\": rpc error: code = NotFound desc = could not find container \"c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67\": container with ID starting with c61e89778307d39b9efef581c32888fe9cd9575265b687529000f41567e6df67 not found: ID does not exist" Sep 10 12:08:45.705372 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705326 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:08:45.705645 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705633 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="config-reloader" Sep 10 12:08:45.705681 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705648 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="config-reloader" Sep 10 12:08:45.705681 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705656 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy-metric" Sep 10 12:08:45.705681 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705662 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy-metric" Sep 10 12:08:45.705681 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705671 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="prom-label-proxy" Sep 10 12:08:45.705681 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705677 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="prom-label-proxy" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705686 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afc67bd2-01b3-4c52-aec2-24943a7b1302" containerName="console" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705691 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="afc67bd2-01b3-4c52-aec2-24943a7b1302" containerName="console" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705704 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy-web" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705710 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy-web" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705717 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="alertmanager" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705722 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="alertmanager" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705729 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="init-config-reloader" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705733 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="init-config-reloader" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705740 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705745 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705789 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705798 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="afc67bd2-01b3-4c52-aec2-24943a7b1302" containerName="console" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705804 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="prom-label-proxy" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705810 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="alertmanager" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705816 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="config-reloader" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705821 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy-metric" Sep 10 12:08:45.705836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.705826 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" containerName="kube-rbac-proxy-web" Sep 10 12:08:45.710536 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.710516 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.713597 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.713570 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 12:08:45.713721 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.713571 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 12:08:45.713721 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.713625 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6r9xp\"" Sep 10 12:08:45.713721 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.713685 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 12:08:45.713988 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.713974 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 12:08:45.714071 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.714027 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 12:08:45.714858 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.714802 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 12:08:45.714858 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.714836 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 12:08:45.714858 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.714849 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 12:08:45.722854 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.722834 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 12:08:45.728587 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.728563 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:08:45.799114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799068 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799114 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799382 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799139 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799382 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799235 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799382 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799276 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799382 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799293 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-config-out\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799382 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799333 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799619 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799388 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799619 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799421 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799619 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799435 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-web-config\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799619 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799470 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799619 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799519 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.799619 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.799553 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvccn\" (UniqueName: \"kubernetes.io/projected/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-kube-api-access-lvccn\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.899965 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.899885 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.899965 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.899922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.899965 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.899944 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900053 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-config-out\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900092 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900119 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900170 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900220 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900198 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-web-config\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900623 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900245 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900623 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900274 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900623 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900299 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvccn\" (UniqueName: \"kubernetes.io/projected/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-kube-api-access-lvccn\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900623 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900390 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900623 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900423 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.900623 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900585 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.901074 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.900940 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.901390 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.901343 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.902316 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.902286 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf17e9fd-0f7d-4dee-8ca9-7555880beab7" path="/var/lib/kubelet/pods/bf17e9fd-0f7d-4dee-8ca9-7555880beab7/volumes" Sep 10 12:08:45.903177 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.903151 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.903270 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.903244 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.903334 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.903318 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.903640 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.903444 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.904202 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.904174 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-config-out\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.904290 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.904244 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.904344 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.904309 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.905468 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.905447 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.905546 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.905488 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-web-config\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:45.910276 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:45.910258 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvccn\" (UniqueName: \"kubernetes.io/projected/4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3-kube-api-access-lvccn\") pod \"alertmanager-main-0\" (UID: \"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:46.021843 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:46.021812 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:08:46.307505 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:46.307340 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:08:46.309913 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:08:46.309887 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9503c8_ae0c_4a4e_bb89_35ff2f53faf3.slice/crio-a9038c40374164e38252581e0c320c83611c0d024a741518dbbb8e25a2ccda0a WatchSource:0}: Error finding container a9038c40374164e38252581e0c320c83611c0d024a741518dbbb8e25a2ccda0a: Status 404 returned error can't find the container with id a9038c40374164e38252581e0c320c83611c0d024a741518dbbb8e25a2ccda0a Sep 10 12:08:46.648267 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:46.648175 2583 generic.go:356] "Generic (PLEG): container finished" podID="4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3" containerID="c863b6862063f7309f8c0574c8306e5e970ce3320cad79f9bacb5c460dda7a84" exitCode=0 Sep 10 12:08:46.648455 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:46.648265 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3","Type":"ContainerDied","Data":"c863b6862063f7309f8c0574c8306e5e970ce3320cad79f9bacb5c460dda7a84"} Sep 10 12:08:46.648455 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:46.648378 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3","Type":"ContainerStarted","Data":"a9038c40374164e38252581e0c320c83611c0d024a741518dbbb8e25a2ccda0a"} Sep 10 12:08:47.656531 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:47.656497 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3","Type":"ContainerStarted","Data":"7f6347796b536ed44fb66381c4598eddce4cb7b9aeecc3af8c0505b435e6aa8d"} Sep 10 12:08:47.656911 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:47.656561 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3","Type":"ContainerStarted","Data":"e153a1670814f42d7471398fcece7cdcd02504dd5f7723b6aa2a5baf519d9169"} Sep 10 12:08:47.656911 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:47.656589 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3","Type":"ContainerStarted","Data":"b1b19b36b0859bd3aae461d35870989fcdb4b038ca7a4be41b05ea2a7aed58c2"} Sep 10 12:08:47.656911 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:47.656614 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3","Type":"ContainerStarted","Data":"7d88b4e674d44e4b3a9e6bed55c90968884fa7ab4b43ef75076f15ea69fe2fde"} Sep 10 12:08:47.656911 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:47.656638 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3","Type":"ContainerStarted","Data":"dcc5499641d7da85600ff8a6ad7dcc549805fc6a1064b0c2bbacdb0d4f460ce7"} Sep 10 12:08:47.656911 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:47.656659 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3","Type":"ContainerStarted","Data":"b5ef35b70de32fb80b2b3b403ea420de3c9467776face4dbea8daceaee9515e8"} Sep 10 12:08:47.686425 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:47.686338 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.671540128 podStartE2EDuration="2.686318336s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.778208ms" isStatefulPod=false podCreationTimestamp="2026-09-10 12:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:08:47.685906619 +0000 UTC m=+148.263230263" watchObservedRunningTime="2026-09-10 12:08:47.686318336 +0000 UTC m=+148.263641961" Sep 10 12:08:48.028169 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.028084 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6bc9f66b54-4mllm"] Sep 10 12:08:48.032665 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.032638 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.035047 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.034813 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 12:08:48.035047 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.034813 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 12:08:48.035047 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.034888 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 12:08:48.035047 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.034893 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 12:08:48.035047 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.034909 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 12:08:48.035580 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.035566 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-m57k7\"" Sep 10 12:08:48.042460 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.042436 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 12:08:48.051140 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.049928 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6bc9f66b54-4mllm"] Sep 10 12:08:48.122869 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.122833 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.123052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.122878 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-secret-telemeter-client\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.123052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.122904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.123052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.122945 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-federate-client-tls\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.123052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.122983 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-telemeter-client-tls\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.123052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.123000 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvhvr\" (UniqueName: \"kubernetes.io/projected/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-kube-api-access-zvhvr\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.123052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.123021 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-serving-certs-ca-bundle\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.123052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.123035 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-metrics-client-ca\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.224155 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.224105 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-secret-telemeter-client\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.224345 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.224174 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.224345 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.224213 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-federate-client-tls\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.224345 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.224280 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-telemeter-client-tls\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.224345 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.224311 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvhvr\" (UniqueName: \"kubernetes.io/projected/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-kube-api-access-zvhvr\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.224345 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.224342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-serving-certs-ca-bundle\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.224618 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.224394 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-metrics-client-ca\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.224618 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.224443 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.225602 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.225571 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.226384 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.226336 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-serving-certs-ca-bundle\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.226488 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.226380 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-metrics-client-ca\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.228942 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.228916 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-federate-client-tls\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.229047 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.229033 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-telemeter-client-tls\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.229490 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.229468 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.229566 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.229532 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-secret-telemeter-client\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.235103 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.235080 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvhvr\" (UniqueName: \"kubernetes.io/projected/91fbbc13-01d5-46c7-95b9-d2f6f56b51af-kube-api-access-zvhvr\") pod \"telemeter-client-6bc9f66b54-4mllm\" (UID: \"91fbbc13-01d5-46c7-95b9-d2f6f56b51af\") " pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.343140 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.343099 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" Sep 10 12:08:48.412062 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.412029 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:08:48.412429 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.412393 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="thanos-sidecar" containerID="cri-o://87c313d308795a4b0a569d4adcdd3b87d52a680119913742c6e7f35679d14b27" gracePeriod=600 Sep 10 12:08:48.412562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.412398 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy" containerID="cri-o://6f6889d03031fa69f1d48d9c389c4d33329f1d6cb47c9addab5c2efa9071e391" gracePeriod=600 Sep 10 12:08:48.412562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.412506 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="config-reloader" containerID="cri-o://2dea52b4e5ca622b3d7f8d8c0f656d38996737739c265134ffa661cbfdb4abd8" gracePeriod=600 Sep 10 12:08:48.412562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.412542 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy-thanos" containerID="cri-o://28b84d07c993649bfafc16ab8d92d7fe544875a32e281cfc915145bd9b0bd14f" gracePeriod=600 Sep 10 12:08:48.412733 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.412392 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="prometheus" containerID="cri-o://454bd4bb6d9e670a264b5d016d435384238cf2ea0fd06bced9a989e9d8f3d20e" gracePeriod=600 Sep 10 12:08:48.412733 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.412591 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy-web" containerID="cri-o://ea574aab11b0c1c86a7625f75d4f2ade642636d2d08be4fda23487496d6f0267" gracePeriod=600 Sep 10 12:08:48.640010 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.638451 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6bc9f66b54-4mllm"] Sep 10 12:08:48.666222 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666190 2583 generic.go:356] "Generic (PLEG): container finished" podID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerID="28b84d07c993649bfafc16ab8d92d7fe544875a32e281cfc915145bd9b0bd14f" exitCode=0 Sep 10 12:08:48.666222 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666218 2583 generic.go:356] "Generic (PLEG): container finished" podID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerID="6f6889d03031fa69f1d48d9c389c4d33329f1d6cb47c9addab5c2efa9071e391" exitCode=0 Sep 10 12:08:48.666222 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666225 2583 generic.go:356] "Generic (PLEG): container finished" podID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerID="ea574aab11b0c1c86a7625f75d4f2ade642636d2d08be4fda23487496d6f0267" exitCode=0 Sep 10 12:08:48.666754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666232 2583 generic.go:356] "Generic (PLEG): container finished" podID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerID="87c313d308795a4b0a569d4adcdd3b87d52a680119913742c6e7f35679d14b27" exitCode=0 Sep 10 12:08:48.666754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666241 2583 generic.go:356] "Generic (PLEG): container finished" podID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerID="2dea52b4e5ca622b3d7f8d8c0f656d38996737739c265134ffa661cbfdb4abd8" exitCode=0 Sep 10 12:08:48.666754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666250 2583 generic.go:356] "Generic (PLEG): container finished" podID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerID="454bd4bb6d9e670a264b5d016d435384238cf2ea0fd06bced9a989e9d8f3d20e" exitCode=0 Sep 10 12:08:48.666754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666253 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerDied","Data":"28b84d07c993649bfafc16ab8d92d7fe544875a32e281cfc915145bd9b0bd14f"} Sep 10 12:08:48.666754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666341 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerDied","Data":"6f6889d03031fa69f1d48d9c389c4d33329f1d6cb47c9addab5c2efa9071e391"} Sep 10 12:08:48.666754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666400 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerDied","Data":"ea574aab11b0c1c86a7625f75d4f2ade642636d2d08be4fda23487496d6f0267"} Sep 10 12:08:48.666754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666438 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerDied","Data":"87c313d308795a4b0a569d4adcdd3b87d52a680119913742c6e7f35679d14b27"} Sep 10 12:08:48.666754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666477 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerDied","Data":"2dea52b4e5ca622b3d7f8d8c0f656d38996737739c265134ffa661cbfdb4abd8"} Sep 10 12:08:48.666754 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.666516 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerDied","Data":"454bd4bb6d9e670a264b5d016d435384238cf2ea0fd06bced9a989e9d8f3d20e"} Sep 10 12:08:48.668715 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.667633 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" event={"ID":"91fbbc13-01d5-46c7-95b9-d2f6f56b51af","Type":"ContainerStarted","Data":"95de7e04c25187b4ddb3f2b1174363b369f9a0885ef376d6a5ecf50534694c57"} Sep 10 12:08:48.671802 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.671783 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:48.829201 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829156 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-metrics-client-ca\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829201 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829206 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-tls-assets\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829201 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829227 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-serving-certs-ca-bundle\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829545 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829247 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-metrics-client-certs\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829545 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829266 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rl7z\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-kube-api-access-4rl7z\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829545 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829288 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829545 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829303 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-1\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829545 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829332 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-config\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829545 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829385 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-2\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829545 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829457 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-kubelet-serving-ca-bundle\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829545 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829505 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-tls\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829545 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829541 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829595 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-web-config\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829643 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-0\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829649 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829669 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-trusted-ca-bundle\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829702 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829730 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-config-out\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829761 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-db\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829800 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-kube-rbac-proxy\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829828 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-thanos-prometheus-http-client-file\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829853 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-grpc-tls\") pod \"5eef4e24-cf25-4965-81d5-6ab964cadd16\" (UID: \"5eef4e24-cf25-4965-81d5-6ab964cadd16\") " Sep 10 12:08:48.829949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829945 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:48.830677 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.829989 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:48.830677 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.830166 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:48.830784 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.830661 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.830784 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.830763 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.830784 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.830778 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-metrics-client-ca\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.830933 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.830794 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.830933 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.830810 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.832371 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.832313 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:08:48.832494 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.832441 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:48.833705 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.832559 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:48.833705 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.832630 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-kube-api-access-4rl7z" (OuterVolumeSpecName: "kube-api-access-4rl7z") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "kube-api-access-4rl7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:08:48.833705 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.832669 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-config-out" (OuterVolumeSpecName: "config-out") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:08:48.833705 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.832839 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:48.834481 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.834104 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-config" (OuterVolumeSpecName: "config") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:48.834481 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.834217 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:48.834822 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.834797 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:08:48.835051 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.835029 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:48.835118 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.835044 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:08:48.835286 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.835267 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:48.835392 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.835374 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:48.836331 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.836312 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:48.846489 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.846430 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-web-config" (OuterVolumeSpecName: "web-config") pod "5eef4e24-cf25-4965-81d5-6ab964cadd16" (UID: "5eef4e24-cf25-4965-81d5-6ab964cadd16"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:08:48.931443 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931396 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-web-config\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931443 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931437 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931443 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931446 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-config-out\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931443 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931455 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-db\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931443 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931463 2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-kube-rbac-proxy\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931474 2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931483 2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-grpc-tls\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931491 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-tls-assets\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931499 2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-metrics-client-certs\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931506 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rl7z\" (UniqueName: \"kubernetes.io/projected/5eef4e24-cf25-4965-81d5-6ab964cadd16-kube-api-access-4rl7z\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931514 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931522 2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-config\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931530 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eef4e24-cf25-4965-81d5-6ab964cadd16-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931538 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:48.931737 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:48.931546 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eef4e24-cf25-4965-81d5-6ab964cadd16-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:08:49.677130 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.677097 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eef4e24-cf25-4965-81d5-6ab964cadd16","Type":"ContainerDied","Data":"2de219bf12a0581dbe8fd70e04a570a7b3c336f6aa574eff42d6002f328c3304"} Sep 10 12:08:49.677675 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.677172 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.677675 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.677187 2583 scope.go:117] "RemoveContainer" containerID="28b84d07c993649bfafc16ab8d92d7fe544875a32e281cfc915145bd9b0bd14f" Sep 10 12:08:49.687204 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.687185 2583 scope.go:117] "RemoveContainer" containerID="6f6889d03031fa69f1d48d9c389c4d33329f1d6cb47c9addab5c2efa9071e391" Sep 10 12:08:49.695247 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.695227 2583 scope.go:117] "RemoveContainer" containerID="ea574aab11b0c1c86a7625f75d4f2ade642636d2d08be4fda23487496d6f0267" Sep 10 12:08:49.702414 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.702387 2583 scope.go:117] "RemoveContainer" containerID="87c313d308795a4b0a569d4adcdd3b87d52a680119913742c6e7f35679d14b27" Sep 10 12:08:49.709944 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.709927 2583 scope.go:117] "RemoveContainer" containerID="2dea52b4e5ca622b3d7f8d8c0f656d38996737739c265134ffa661cbfdb4abd8" Sep 10 12:08:49.711091 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.710983 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:08:49.717659 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.717637 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:08:49.718658 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.718630 2583 scope.go:117] "RemoveContainer" containerID="454bd4bb6d9e670a264b5d016d435384238cf2ea0fd06bced9a989e9d8f3d20e" Sep 10 12:08:49.728062 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.728036 2583 scope.go:117] "RemoveContainer" containerID="89694fcc4e467b972622cc171d37382f26c93128c4940645f59248e3f271f1e3" Sep 10 12:08:49.759399 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759366 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:08:49.759848 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759830 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy-web" Sep 10 12:08:49.759926 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759853 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy-web" Sep 10 12:08:49.759926 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759868 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="init-config-reloader" Sep 10 12:08:49.759926 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759874 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="init-config-reloader" Sep 10 12:08:49.759926 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759882 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="config-reloader" Sep 10 12:08:49.759926 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759891 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="config-reloader" Sep 10 12:08:49.759926 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759914 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="thanos-sidecar" Sep 10 12:08:49.759926 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759925 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="thanos-sidecar" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759934 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy-thanos" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759939 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy-thanos" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759947 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759951 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759960 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="prometheus" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.759967 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="prometheus" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.760030 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="prometheus" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.760036 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="config-reloader" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.760044 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy-web" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.760050 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.760072 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="kube-rbac-proxy-thanos" Sep 10 12:08:49.760313 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.760084 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" containerName="thanos-sidecar" Sep 10 12:08:49.765963 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.765935 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.770506 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.770483 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 12:08:49.770673 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.770648 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mfz87\"" Sep 10 12:08:49.771103 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.771081 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 12:08:49.772016 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.771997 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 12:08:49.772329 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.772117 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 12:08:49.772329 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.772134 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 12:08:49.772329 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.772243 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 12:08:49.772329 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.772294 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 12:08:49.772600 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.772573 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fo0762qlveoan\"" Sep 10 12:08:49.772732 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.772705 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 12:08:49.773043 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.773021 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 12:08:49.773503 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.773477 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 12:08:49.773729 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.773710 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 12:08:49.778376 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.778339 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 12:08:49.780758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.780739 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 12:08:49.798319 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.798280 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 12:08:49.798519 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.798416 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:08:49.906896 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.906860 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eef4e24-cf25-4965-81d5-6ab964cadd16" path="/var/lib/kubelet/pods/5eef4e24-cf25-4965-81d5-6ab964cadd16/volumes" Sep 10 12:08:49.938745 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.938673 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.938745 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.938726 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.938875 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.938762 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.938875 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.938787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.938875 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.938808 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/22df79fa-6ca3-4868-b1f9-aaebd68032b3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.938875 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.938827 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.938998 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.938904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.938998 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.938942 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk7cz\" (UniqueName: \"kubernetes.io/projected/22df79fa-6ca3-4868-b1f9-aaebd68032b3-kube-api-access-sk7cz\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.938998 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.938965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/22df79fa-6ca3-4868-b1f9-aaebd68032b3-config-out\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939087 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939026 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939087 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939044 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939087 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939060 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939087 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939083 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939215 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939100 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939215 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939125 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939215 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939180 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-config\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939215 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939201 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939330 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939224 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939330 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939247 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:49.939330 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:49.939262 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-web-config\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040578 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040530 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040578 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040585 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk7cz\" (UniqueName: \"kubernetes.io/projected/22df79fa-6ca3-4868-b1f9-aaebd68032b3-kube-api-access-sk7cz\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040822 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040616 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/22df79fa-6ca3-4868-b1f9-aaebd68032b3-config-out\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040822 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040651 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040822 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040673 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040822 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040697 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040822 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040822 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040822 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040780 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.040822 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040812 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-config\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040841 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040873 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040904 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040928 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-web-config\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040959 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.040995 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.041005 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.041060 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.041094 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.041128 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/22df79fa-6ca3-4868-b1f9-aaebd68032b3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041259 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.041153 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.041988 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.041815 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.042390 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.042343 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.043058 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.042638 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.043994 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.043972 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/22df79fa-6ca3-4868-b1f9-aaebd68032b3-config-out\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.044549 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.044506 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.044549 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.044524 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.045185 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.045114 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.045185 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.045159 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.045809 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.045785 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-web-config\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.045892 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.045786 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.046590 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.046505 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.046590 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.046536 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-config\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.047283 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.047214 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.047574 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.047553 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.048330 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.048168 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/22df79fa-6ca3-4868-b1f9-aaebd68032b3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.048330 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.048294 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.049030 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.049008 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/22df79fa-6ca3-4868-b1f9-aaebd68032b3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.049114 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.049065 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/22df79fa-6ca3-4868-b1f9-aaebd68032b3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.051986 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.051968 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk7cz\" (UniqueName: \"kubernetes.io/projected/22df79fa-6ca3-4868-b1f9-aaebd68032b3-kube-api-access-sk7cz\") pod \"prometheus-k8s-0\" (UID: \"22df79fa-6ca3-4868-b1f9-aaebd68032b3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.077933 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.077893 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:08:50.386001 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.385950 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:08:50.605883 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:08:50.605848 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22df79fa_6ca3_4868_b1f9_aaebd68032b3.slice/crio-dcace0a99067806d6d0001de44b576990ab777861766d98d605cfaedff2c95cd WatchSource:0}: Error finding container dcace0a99067806d6d0001de44b576990ab777861766d98d605cfaedff2c95cd: Status 404 returned error can't find the container with id dcace0a99067806d6d0001de44b576990ab777861766d98d605cfaedff2c95cd Sep 10 12:08:50.683861 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.683830 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22df79fa-6ca3-4868-b1f9-aaebd68032b3","Type":"ContainerStarted","Data":"ca880a9cc85681a1801c6fa1a7bd0bdd35e9e9e7e03b56f3c2c54b025837a4b5"} Sep 10 12:08:50.684240 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:50.683902 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22df79fa-6ca3-4868-b1f9-aaebd68032b3","Type":"ContainerStarted","Data":"dcace0a99067806d6d0001de44b576990ab777861766d98d605cfaedff2c95cd"} Sep 10 12:08:51.690550 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:51.690512 2583 generic.go:356] "Generic (PLEG): container finished" podID="22df79fa-6ca3-4868-b1f9-aaebd68032b3" containerID="ca880a9cc85681a1801c6fa1a7bd0bdd35e9e9e7e03b56f3c2c54b025837a4b5" exitCode=0 Sep 10 12:08:51.690965 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:51.690595 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22df79fa-6ca3-4868-b1f9-aaebd68032b3","Type":"ContainerDied","Data":"ca880a9cc85681a1801c6fa1a7bd0bdd35e9e9e7e03b56f3c2c54b025837a4b5"} Sep 10 12:08:51.692924 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:51.692897 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" event={"ID":"91fbbc13-01d5-46c7-95b9-d2f6f56b51af","Type":"ContainerStarted","Data":"160833035d8dca6b79b621e6954b07b94d3d1458bbb0f03097b43e6c76619389"} Sep 10 12:08:51.693007 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:51.692950 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" event={"ID":"91fbbc13-01d5-46c7-95b9-d2f6f56b51af","Type":"ContainerStarted","Data":"f803d0f365be54f672c143709e983632bceaff4ce75fa6588c9da7a9107fedea"} Sep 10 12:08:51.693007 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:51.692969 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" event={"ID":"91fbbc13-01d5-46c7-95b9-d2f6f56b51af","Type":"ContainerStarted","Data":"6affd019411eb14d146c2d900422ddce5076f0f4f825b4bb40b4f0d047a9b253"} Sep 10 12:08:51.744140 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:51.744090 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6bc9f66b54-4mllm" podStartSLOduration=1.7313546020000001 podStartE2EDuration="3.744073615s" totalImagesPullingTime="2.012719013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:08:51.74216929 +0000 UTC m=+152.319492905" watchObservedRunningTime="2026-09-10 12:08:51.744073615 +0000 UTC m=+152.321397223" Sep 10 12:08:52.700330 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:52.700293 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22df79fa-6ca3-4868-b1f9-aaebd68032b3","Type":"ContainerStarted","Data":"43cb98f0828860c150202b51158a7412e456ee9b5035089cbd1add5fed021b4a"} Sep 10 12:08:52.700727 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:52.700377 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22df79fa-6ca3-4868-b1f9-aaebd68032b3","Type":"ContainerStarted","Data":"afaedd1c7ccfd2669c391958950d8714e06e5042d2330fb6ba113e745d6342bb"} Sep 10 12:08:52.700727 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:52.700405 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22df79fa-6ca3-4868-b1f9-aaebd68032b3","Type":"ContainerStarted","Data":"a248d17e1dc966066f796128897db0ab3106e276cdb1bdb3e2a73756f4bb9d86"} Sep 10 12:08:52.700727 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:52.700430 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22df79fa-6ca3-4868-b1f9-aaebd68032b3","Type":"ContainerStarted","Data":"99286295837c95cd4db155f548306f6305bb6b679ab94557d553952c469e4c7d"} Sep 10 12:08:52.700727 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:52.700453 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22df79fa-6ca3-4868-b1f9-aaebd68032b3","Type":"ContainerStarted","Data":"a30b15d54cc60e74dc8cc5e75703b171c43af2fba901f1c02e48b36e5dac3710"} Sep 10 12:08:52.700727 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:52.700481 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22df79fa-6ca3-4868-b1f9-aaebd68032b3","Type":"ContainerStarted","Data":"9799fe3bd8092840b11784c4ff395634a92c87e64b8232555eba59f552b7333f"} Sep 10 12:08:52.730877 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:52.730819 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.710203967 podStartE2EDuration="3.730800272s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.596305ms" isStatefulPod=false podCreationTimestamp="2026-09-10 12:08:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:08:52.729005405 +0000 UTC m=+153.306329037" watchObservedRunningTime="2026-09-10 12:08:52.730800272 +0000 UTC m=+153.308123882" Sep 10 12:08:55.078167 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:08:55.078125 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:09:50.078846 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:09:50.078805 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:09:50.095226 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:09:50.095194 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:09:50.954666 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:09:50.954640 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:11:19.780952 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:19.780927 2583 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 12:11:31.048888 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.048850 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p27xp"] Sep 10 12:11:31.052121 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.052105 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" Sep 10 12:11:31.058520 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.058099 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 10 12:11:31.058520 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.058259 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7sdx7\"" Sep 10 12:11:31.110516 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.110483 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 10 12:11:31.126369 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.126319 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p27xp"] Sep 10 12:11:31.142719 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.142685 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3e47b9d-dc18-49a6-b80f-73f6650febad-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p27xp\" (UID: \"c3e47b9d-dc18-49a6-b80f-73f6650febad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" Sep 10 12:11:31.142882 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.142732 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ksxf\" (UniqueName: \"kubernetes.io/projected/c3e47b9d-dc18-49a6-b80f-73f6650febad-kube-api-access-5ksxf\") pod \"cert-manager-cainjector-559cdd8c87-p27xp\" (UID: \"c3e47b9d-dc18-49a6-b80f-73f6650febad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" Sep 10 12:11:31.243790 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.243751 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3e47b9d-dc18-49a6-b80f-73f6650febad-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p27xp\" (UID: \"c3e47b9d-dc18-49a6-b80f-73f6650febad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" Sep 10 12:11:31.244002 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.243810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ksxf\" (UniqueName: \"kubernetes.io/projected/c3e47b9d-dc18-49a6-b80f-73f6650febad-kube-api-access-5ksxf\") pod \"cert-manager-cainjector-559cdd8c87-p27xp\" (UID: \"c3e47b9d-dc18-49a6-b80f-73f6650febad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" Sep 10 12:11:31.281942 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.281907 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ksxf\" (UniqueName: \"kubernetes.io/projected/c3e47b9d-dc18-49a6-b80f-73f6650febad-kube-api-access-5ksxf\") pod \"cert-manager-cainjector-559cdd8c87-p27xp\" (UID: \"c3e47b9d-dc18-49a6-b80f-73f6650febad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" Sep 10 12:11:31.305780 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.305710 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3e47b9d-dc18-49a6-b80f-73f6650febad-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p27xp\" (UID: \"c3e47b9d-dc18-49a6-b80f-73f6650febad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" Sep 10 12:11:31.372944 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.372904 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" Sep 10 12:11:31.664184 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.664088 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p27xp"] Sep 10 12:11:31.667175 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:11:31.667149 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3e47b9d_dc18_49a6_b80f_73f6650febad.slice/crio-e55ec55229d232358709bb48c8e23d23ace889036cb99fb00f529937d5cae784 WatchSource:0}: Error finding container e55ec55229d232358709bb48c8e23d23ace889036cb99fb00f529937d5cae784: Status 404 returned error can't find the container with id e55ec55229d232358709bb48c8e23d23ace889036cb99fb00f529937d5cae784 Sep 10 12:11:31.669239 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:31.669221 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:11:32.353111 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:32.353075 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" event={"ID":"c3e47b9d-dc18-49a6-b80f-73f6650febad","Type":"ContainerStarted","Data":"e55ec55229d232358709bb48c8e23d23ace889036cb99fb00f529937d5cae784"} Sep 10 12:11:35.376552 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:35.376506 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" event={"ID":"c3e47b9d-dc18-49a6-b80f-73f6650febad","Type":"ContainerStarted","Data":"3b9bc7f14d479201c97121058a70ec763d058218ff3c56c232ce23253c2f6f07"} Sep 10 12:11:40.357777 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.357719 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p27xp" podStartSLOduration=7.205433203 podStartE2EDuration="10.357700109s" totalImagesPullingTime="3.152266906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:11:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:11:35.416653668 +0000 UTC m=+315.993977277" watchObservedRunningTime="2026-09-10 12:11:40.357700109 +0000 UTC m=+320.935023717" Sep 10 12:11:40.358715 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.358698 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kpjb7"] Sep 10 12:11:40.362138 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.362121 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kpjb7" Sep 10 12:11:40.364906 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.364889 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-86vmm\"" Sep 10 12:11:40.425413 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.425347 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxz2f\" (UniqueName: \"kubernetes.io/projected/019462b5-9a80-455b-b54b-6ee1f0b1015d-kube-api-access-mxz2f\") pod \"cert-manager-55b4d954d5-kpjb7\" (UID: \"019462b5-9a80-455b-b54b-6ee1f0b1015d\") " pod="cert-manager/cert-manager-55b4d954d5-kpjb7" Sep 10 12:11:40.425578 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.425477 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/019462b5-9a80-455b-b54b-6ee1f0b1015d-bound-sa-token\") pod \"cert-manager-55b4d954d5-kpjb7\" (UID: \"019462b5-9a80-455b-b54b-6ee1f0b1015d\") " pod="cert-manager/cert-manager-55b4d954d5-kpjb7" Sep 10 12:11:40.431232 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.431191 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kpjb7"] Sep 10 12:11:40.526045 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.525989 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxz2f\" (UniqueName: \"kubernetes.io/projected/019462b5-9a80-455b-b54b-6ee1f0b1015d-kube-api-access-mxz2f\") pod \"cert-manager-55b4d954d5-kpjb7\" (UID: \"019462b5-9a80-455b-b54b-6ee1f0b1015d\") " pod="cert-manager/cert-manager-55b4d954d5-kpjb7" Sep 10 12:11:40.526225 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.526087 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/019462b5-9a80-455b-b54b-6ee1f0b1015d-bound-sa-token\") pod \"cert-manager-55b4d954d5-kpjb7\" (UID: \"019462b5-9a80-455b-b54b-6ee1f0b1015d\") " pod="cert-manager/cert-manager-55b4d954d5-kpjb7" Sep 10 12:11:40.539569 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.539532 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/019462b5-9a80-455b-b54b-6ee1f0b1015d-bound-sa-token\") pod \"cert-manager-55b4d954d5-kpjb7\" (UID: \"019462b5-9a80-455b-b54b-6ee1f0b1015d\") " pod="cert-manager/cert-manager-55b4d954d5-kpjb7" Sep 10 12:11:40.539724 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.539707 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxz2f\" (UniqueName: \"kubernetes.io/projected/019462b5-9a80-455b-b54b-6ee1f0b1015d-kube-api-access-mxz2f\") pod \"cert-manager-55b4d954d5-kpjb7\" (UID: \"019462b5-9a80-455b-b54b-6ee1f0b1015d\") " pod="cert-manager/cert-manager-55b4d954d5-kpjb7" Sep 10 12:11:40.670922 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.670836 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kpjb7" Sep 10 12:11:40.945892 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:40.945818 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kpjb7"] Sep 10 12:11:40.948897 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:11:40.948870 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod019462b5_9a80_455b_b54b_6ee1f0b1015d.slice/crio-68a01624dcc1060e630bed7a0fbba99bbb7b781496e8a82c953cb0bd6ace45ab WatchSource:0}: Error finding container 68a01624dcc1060e630bed7a0fbba99bbb7b781496e8a82c953cb0bd6ace45ab: Status 404 returned error can't find the container with id 68a01624dcc1060e630bed7a0fbba99bbb7b781496e8a82c953cb0bd6ace45ab Sep 10 12:11:41.405310 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:41.405280 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kpjb7" event={"ID":"019462b5-9a80-455b-b54b-6ee1f0b1015d","Type":"ContainerStarted","Data":"6f4ba0196930bea43e2fd7c667f519f0aa68ab8bf57d492049079603b259e38f"} Sep 10 12:11:41.405310 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:41.405324 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kpjb7" event={"ID":"019462b5-9a80-455b-b54b-6ee1f0b1015d","Type":"ContainerStarted","Data":"68a01624dcc1060e630bed7a0fbba99bbb7b781496e8a82c953cb0bd6ace45ab"} Sep 10 12:11:41.430405 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:11:41.430334 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-kpjb7" podStartSLOduration=1.430316238 podStartE2EDuration="1.430316238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:11:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:11:41.429191284 +0000 UTC m=+322.006514893" watchObservedRunningTime="2026-09-10 12:11:41.430316238 +0000 UTC m=+322.007639847" Sep 10 12:12:03.730375 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.730315 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw"] Sep 10 12:12:03.733843 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.733824 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.737933 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.737907 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:12:03.737933 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.737923 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 12:12:03.738153 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.737923 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 12:12:03.738153 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.737968 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-s4k95\"" Sep 10 12:12:03.738153 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.737907 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 12:12:03.738153 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.737960 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 12:12:03.748038 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.748016 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw"] Sep 10 12:12:03.834407 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.834369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3e6d498-0777-4a32-9821-d043d1575448-metrics-cert\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.834618 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.834476 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e3e6d498-0777-4a32-9821-d043d1575448-manager-config\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.834618 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.834511 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkzfn\" (UniqueName: \"kubernetes.io/projected/e3e6d498-0777-4a32-9821-d043d1575448-kube-api-access-tkzfn\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.834618 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.834547 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3e6d498-0777-4a32-9821-d043d1575448-cert\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.935871 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.935836 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e3e6d498-0777-4a32-9821-d043d1575448-manager-config\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.935871 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.935880 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkzfn\" (UniqueName: \"kubernetes.io/projected/e3e6d498-0777-4a32-9821-d043d1575448-kube-api-access-tkzfn\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.936137 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.935912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3e6d498-0777-4a32-9821-d043d1575448-cert\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.936137 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.935938 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3e6d498-0777-4a32-9821-d043d1575448-metrics-cert\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.936567 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.936545 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e3e6d498-0777-4a32-9821-d043d1575448-manager-config\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.938598 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.938577 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3e6d498-0777-4a32-9821-d043d1575448-metrics-cert\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.938679 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.938618 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3e6d498-0777-4a32-9821-d043d1575448-cert\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:03.952199 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:03.952171 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkzfn\" (UniqueName: \"kubernetes.io/projected/e3e6d498-0777-4a32-9821-d043d1575448-kube-api-access-tkzfn\") pod \"lws-controller-manager-5b86f6cfc4-2k4dw\" (UID: \"e3e6d498-0777-4a32-9821-d043d1575448\") " pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:04.043038 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:04.043001 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:04.326612 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:04.326583 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw"] Sep 10 12:12:04.328665 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:12:04.328639 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e6d498_0777_4a32_9821_d043d1575448.slice/crio-95445166705f6db42b091c38e8478bf705fa1d10c96d3e3f55fde159ed8d1895 WatchSource:0}: Error finding container 95445166705f6db42b091c38e8478bf705fa1d10c96d3e3f55fde159ed8d1895: Status 404 returned error can't find the container with id 95445166705f6db42b091c38e8478bf705fa1d10c96d3e3f55fde159ed8d1895 Sep 10 12:12:04.502656 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:04.502621 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" event={"ID":"e3e6d498-0777-4a32-9821-d043d1575448","Type":"ContainerStarted","Data":"95445166705f6db42b091c38e8478bf705fa1d10c96d3e3f55fde159ed8d1895"} Sep 10 12:12:05.290466 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.290427 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp"] Sep 10 12:12:05.295262 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.295243 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.298796 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.298772 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 12:12:05.299156 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.299139 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 12:12:05.299332 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.299314 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 12:12:05.299607 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.299590 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rzdvf\"" Sep 10 12:12:05.299758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.299740 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 12:12:05.313859 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.313835 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp"] Sep 10 12:12:05.450878 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.450843 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.451052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.450898 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcvm6\" (UniqueName: \"kubernetes.io/projected/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-kube-api-access-hcvm6\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.451052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.450975 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-tmp\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.451052 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.451040 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-manifests\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.451155 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.451089 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.552540 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.552453 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.552540 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.552506 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcvm6\" (UniqueName: \"kubernetes.io/projected/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-kube-api-access-hcvm6\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.552767 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.552548 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-tmp\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.552767 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.552605 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-manifests\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.552767 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.552717 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.553066 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.553043 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-manifests\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.555711 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.555673 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-tmp\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.555937 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.555917 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.556091 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.556071 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.565683 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.565651 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcvm6\" (UniqueName: \"kubernetes.io/projected/e1b992a3-2e13-4b3e-8781-bb86fdcc1a79-kube-api-access-hcvm6\") pod \"opendatahub-operator-controller-manager-7d9854688c-lxdsp\" (UID: \"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79\") " pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.606388 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.606328 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:05.918386 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:05.918335 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp"] Sep 10 12:12:05.922377 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:12:05.922330 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b992a3_2e13_4b3e_8781_bb86fdcc1a79.slice/crio-cc3ffbd69f54f3431d65a89e859c55ce4f6eebc975727f32c7b6ac4462b5476f WatchSource:0}: Error finding container cc3ffbd69f54f3431d65a89e859c55ce4f6eebc975727f32c7b6ac4462b5476f: Status 404 returned error can't find the container with id cc3ffbd69f54f3431d65a89e859c55ce4f6eebc975727f32c7b6ac4462b5476f Sep 10 12:12:06.518479 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:06.518444 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" event={"ID":"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79","Type":"ContainerStarted","Data":"cc3ffbd69f54f3431d65a89e859c55ce4f6eebc975727f32c7b6ac4462b5476f"} Sep 10 12:12:08.530315 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:08.530278 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" event={"ID":"e3e6d498-0777-4a32-9821-d043d1575448","Type":"ContainerStarted","Data":"aef354e5214b2bc028d493da2584dc02ea1c8e679706eb1b31ea0751d636634a"} Sep 10 12:12:08.530751 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:08.530365 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:08.550338 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:08.550275 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" podStartSLOduration=2.315149697 podStartE2EDuration="5.550251522s" totalImagesPullingTime="3.235101825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:12:08.549079142 +0000 UTC m=+349.126402752" watchObservedRunningTime="2026-09-10 12:12:08.550251522 +0000 UTC m=+349.127575132" Sep 10 12:12:11.553075 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:11.553039 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" event={"ID":"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79","Type":"ContainerStarted","Data":"99bdb8518734db26b26cf8e1f07ee6740d3d7bb0f50e037e16db393e01828570"} Sep 10 12:12:12.559536 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:12.559504 2583 generic.go:356] "Generic (PLEG): container finished" podID="e1b992a3-2e13-4b3e-8781-bb86fdcc1a79" containerID="99bdb8518734db26b26cf8e1f07ee6740d3d7bb0f50e037e16db393e01828570" exitCode=0 Sep 10 12:12:12.559974 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:12.559589 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" event={"ID":"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79","Type":"ContainerDied","Data":"99bdb8518734db26b26cf8e1f07ee6740d3d7bb0f50e037e16db393e01828570"} Sep 10 12:12:13.567232 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:13.567194 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" event={"ID":"e1b992a3-2e13-4b3e-8781-bb86fdcc1a79","Type":"ContainerStarted","Data":"7d3292843df43382edb059151f3f1adb7594c68377ab2c28dc9939229e5d9494"} Sep 10 12:12:13.567655 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:13.567284 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:13.593740 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:13.593678 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" podStartSLOduration=2.244822491 podStartE2EDuration="8.593654892s" totalImagesPullingTime="5.261844435s" totalInitContainerRuntime="1.086987966s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:12:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:12:13.590929636 +0000 UTC m=+354.168253246" watchObservedRunningTime="2026-09-10 12:12:13.593654892 +0000 UTC m=+354.170978504" Sep 10 12:12:19.538489 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:19.538459 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b86f6cfc4-2k4dw" Sep 10 12:12:24.574164 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:24.574131 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d9854688c-lxdsp" Sep 10 12:12:26.696509 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.696471 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp"] Sep 10 12:12:26.705285 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.705256 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:26.708547 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.708522 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 10 12:12:26.708653 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.708562 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wjdn4\"" Sep 10 12:12:26.709082 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.709067 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 12:12:26.709126 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.709087 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 12:12:26.712917 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.712901 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 10 12:12:26.728706 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.728677 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp"] Sep 10 12:12:26.753408 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.753376 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b819d774-1d7e-4079-89be-cc8e15a64098-tmp\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:26.753654 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.753544 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdmff\" (UniqueName: \"kubernetes.io/projected/b819d774-1d7e-4079-89be-cc8e15a64098-kube-api-access-hdmff\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:26.753654 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.753617 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b819d774-1d7e-4079-89be-cc8e15a64098-tls-certs\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:26.855023 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.854986 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b819d774-1d7e-4079-89be-cc8e15a64098-tmp\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:26.855186 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.855069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdmff\" (UniqueName: \"kubernetes.io/projected/b819d774-1d7e-4079-89be-cc8e15a64098-kube-api-access-hdmff\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:26.855186 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.855119 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b819d774-1d7e-4079-89be-cc8e15a64098-tls-certs\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:26.855271 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:12:26.855249 2583 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 10 12:12:26.855338 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:12:26.855326 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b819d774-1d7e-4079-89be-cc8e15a64098-tls-certs podName:b819d774-1d7e-4079-89be-cc8e15a64098 nodeName:}" failed. No retries permitted until 2026-09-10 12:12:27.355305608 +0000 UTC m=+367.932629198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b819d774-1d7e-4079-89be-cc8e15a64098-tls-certs") pod "kube-auth-proxy-64d8784dff-k7lwp" (UID: "b819d774-1d7e-4079-89be-cc8e15a64098") : secret "kube-auth-proxy-tls" not found Sep 10 12:12:26.857441 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.857414 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b819d774-1d7e-4079-89be-cc8e15a64098-tmp\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:26.869141 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:26.869115 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdmff\" (UniqueName: \"kubernetes.io/projected/b819d774-1d7e-4079-89be-cc8e15a64098-kube-api-access-hdmff\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:27.359480 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:27.359437 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b819d774-1d7e-4079-89be-cc8e15a64098-tls-certs\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:27.362082 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:27.362058 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b819d774-1d7e-4079-89be-cc8e15a64098-tls-certs\") pod \"kube-auth-proxy-64d8784dff-k7lwp\" (UID: \"b819d774-1d7e-4079-89be-cc8e15a64098\") " pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:27.615639 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:27.615539 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" Sep 10 12:12:27.902536 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:12:27.902494 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb819d774_1d7e_4079_89be_cc8e15a64098.slice/crio-f0d20161cda48694c5b6293c1f6e9bc31f6586a3af3c8a3c2b3362b2777c3148 WatchSource:0}: Error finding container f0d20161cda48694c5b6293c1f6e9bc31f6586a3af3c8a3c2b3362b2777c3148: Status 404 returned error can't find the container with id f0d20161cda48694c5b6293c1f6e9bc31f6586a3af3c8a3c2b3362b2777c3148 Sep 10 12:12:27.903343 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:27.903319 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp"] Sep 10 12:12:28.638244 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:28.638196 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" event={"ID":"b819d774-1d7e-4079-89be-cc8e15a64098","Type":"ContainerStarted","Data":"f0d20161cda48694c5b6293c1f6e9bc31f6586a3af3c8a3c2b3362b2777c3148"} Sep 10 12:12:31.654607 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:31.654571 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" event={"ID":"b819d774-1d7e-4079-89be-cc8e15a64098","Type":"ContainerStarted","Data":"9d753c2b6141c460cb4975a66898f6c1a5e78f4c374f225a8e1ca08ce19eb270"} Sep 10 12:12:31.671225 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:12:31.671172 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64d8784dff-k7lwp" podStartSLOduration=2.877824582 podStartE2EDuration="5.671152712s" totalImagesPullingTime="2.79332813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:12:31.670587043 +0000 UTC m=+372.247910653" watchObservedRunningTime="2026-09-10 12:12:31.671152712 +0000 UTC m=+372.248476322" Sep 10 12:14:19.167741 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.167705 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp"] Sep 10 12:14:19.171370 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.171334 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:19.174303 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.174280 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6hxl4\"" Sep 10 12:14:19.174450 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.174284 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 12:14:19.175309 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.175283 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 12:14:19.181442 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.181422 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6b306f6-7c17-4a37-864a-5d2fb22a759a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-62rlp\" (UID: \"f6b306f6-7c17-4a37-864a-5d2fb22a759a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:19.181525 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.181472 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cdgd\" (UniqueName: \"kubernetes.io/projected/f6b306f6-7c17-4a37-864a-5d2fb22a759a-kube-api-access-7cdgd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-62rlp\" (UID: \"f6b306f6-7c17-4a37-864a-5d2fb22a759a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:19.208573 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.208533 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp"] Sep 10 12:14:19.282600 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.282552 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cdgd\" (UniqueName: \"kubernetes.io/projected/f6b306f6-7c17-4a37-864a-5d2fb22a759a-kube-api-access-7cdgd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-62rlp\" (UID: \"f6b306f6-7c17-4a37-864a-5d2fb22a759a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:19.282805 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.282665 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6b306f6-7c17-4a37-864a-5d2fb22a759a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-62rlp\" (UID: \"f6b306f6-7c17-4a37-864a-5d2fb22a759a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:19.283069 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.283046 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6b306f6-7c17-4a37-864a-5d2fb22a759a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-62rlp\" (UID: \"f6b306f6-7c17-4a37-864a-5d2fb22a759a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:19.294549 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.294524 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cdgd\" (UniqueName: \"kubernetes.io/projected/f6b306f6-7c17-4a37-864a-5d2fb22a759a-kube-api-access-7cdgd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-62rlp\" (UID: \"f6b306f6-7c17-4a37-864a-5d2fb22a759a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:19.481810 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.481709 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:19.767113 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:19.767086 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp"] Sep 10 12:14:19.769894 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:14:19.769865 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b306f6_7c17_4a37_864a_5d2fb22a759a.slice/crio-78c7e7b5980154685070a7442b6c5fce1924c7f26adb320593db26d94e00c526 WatchSource:0}: Error finding container 78c7e7b5980154685070a7442b6c5fce1924c7f26adb320593db26d94e00c526: Status 404 returned error can't find the container with id 78c7e7b5980154685070a7442b6c5fce1924c7f26adb320593db26d94e00c526 Sep 10 12:14:20.158325 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:20.158288 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" event={"ID":"f6b306f6-7c17-4a37-864a-5d2fb22a759a","Type":"ContainerStarted","Data":"78c7e7b5980154685070a7442b6c5fce1924c7f26adb320593db26d94e00c526"} Sep 10 12:14:24.183641 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:24.183603 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" event={"ID":"f6b306f6-7c17-4a37-864a-5d2fb22a759a","Type":"ContainerStarted","Data":"5014074967d8541c84c16494d190aabc380cfd64942fb09dfdad32196513f4bc"} Sep 10 12:14:24.184042 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:24.183662 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:24.215246 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:24.215181 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" podStartSLOduration=1.491499635 podStartE2EDuration="5.215160682s" totalImagesPullingTime="3.723661047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:14:24.212593969 +0000 UTC m=+484.789917577" watchObservedRunningTime="2026-09-10 12:14:24.215160682 +0000 UTC m=+484.792484292" Sep 10 12:14:35.190312 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:35.190281 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-62rlp" Sep 10 12:14:58.344330 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:58.344285 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sxd2v"] Sep 10 12:14:58.347939 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:58.347914 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" Sep 10 12:14:58.350333 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:58.350312 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-glkkw\"" Sep 10 12:14:58.356096 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:58.356073 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sxd2v"] Sep 10 12:14:58.448883 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:58.448784 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkz5k\" (UniqueName: \"kubernetes.io/projected/a0178d3e-ece3-438b-83f5-bbe2da743aa0-kube-api-access-gkz5k\") pod \"authorino-7bcfd8f9cd-sxd2v\" (UID: \"a0178d3e-ece3-438b-83f5-bbe2da743aa0\") " pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" Sep 10 12:14:58.550018 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:58.549983 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkz5k\" (UniqueName: \"kubernetes.io/projected/a0178d3e-ece3-438b-83f5-bbe2da743aa0-kube-api-access-gkz5k\") pod \"authorino-7bcfd8f9cd-sxd2v\" (UID: \"a0178d3e-ece3-438b-83f5-bbe2da743aa0\") " pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" Sep 10 12:14:58.560947 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:58.560916 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkz5k\" (UniqueName: \"kubernetes.io/projected/a0178d3e-ece3-438b-83f5-bbe2da743aa0-kube-api-access-gkz5k\") pod \"authorino-7bcfd8f9cd-sxd2v\" (UID: \"a0178d3e-ece3-438b-83f5-bbe2da743aa0\") " pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" Sep 10 12:14:58.657147 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:58.657102 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" Sep 10 12:14:58.942525 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:58.942498 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sxd2v"] Sep 10 12:14:59.353757 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:14:59.353724 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" event={"ID":"a0178d3e-ece3-438b-83f5-bbe2da743aa0","Type":"ContainerStarted","Data":"bdc76a595073711cbaab34eb78f16278aad8906cbde528f81ac024ee63d7dd10"} Sep 10 12:15:01.365278 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:01.365237 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" event={"ID":"a0178d3e-ece3-438b-83f5-bbe2da743aa0","Type":"ContainerStarted","Data":"a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7"} Sep 10 12:15:01.387882 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:01.387834 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" podStartSLOduration=1.432971209 podStartE2EDuration="3.387813368s" totalImagesPullingTime="1.954842159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:01.384149711 +0000 UTC m=+521.961473334" watchObservedRunningTime="2026-09-10 12:15:01.387813368 +0000 UTC m=+521.965136977" Sep 10 12:15:01.439788 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:01.439754 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sxd2v"] Sep 10 12:15:03.375291 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:03.375252 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" podUID="a0178d3e-ece3-438b-83f5-bbe2da743aa0" containerName="authorino" containerID="cri-o://a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7" gracePeriod=30 Sep 10 12:15:03.617852 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:03.617825 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" Sep 10 12:15:03.690066 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:03.689972 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkz5k\" (UniqueName: \"kubernetes.io/projected/a0178d3e-ece3-438b-83f5-bbe2da743aa0-kube-api-access-gkz5k\") pod \"a0178d3e-ece3-438b-83f5-bbe2da743aa0\" (UID: \"a0178d3e-ece3-438b-83f5-bbe2da743aa0\") " Sep 10 12:15:03.692319 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:03.692291 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0178d3e-ece3-438b-83f5-bbe2da743aa0-kube-api-access-gkz5k" (OuterVolumeSpecName: "kube-api-access-gkz5k") pod "a0178d3e-ece3-438b-83f5-bbe2da743aa0" (UID: "a0178d3e-ece3-438b-83f5-bbe2da743aa0"). InnerVolumeSpecName "kube-api-access-gkz5k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:15:03.791260 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:03.791220 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkz5k\" (UniqueName: \"kubernetes.io/projected/a0178d3e-ece3-438b-83f5-bbe2da743aa0-kube-api-access-gkz5k\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:15:04.380514 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:04.380476 2583 generic.go:356] "Generic (PLEG): container finished" podID="a0178d3e-ece3-438b-83f5-bbe2da743aa0" containerID="a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7" exitCode=0 Sep 10 12:15:04.380514 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:04.380520 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" Sep 10 12:15:04.381032 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:04.380569 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" event={"ID":"a0178d3e-ece3-438b-83f5-bbe2da743aa0","Type":"ContainerDied","Data":"a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7"} Sep 10 12:15:04.381032 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:04.380616 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-sxd2v" event={"ID":"a0178d3e-ece3-438b-83f5-bbe2da743aa0","Type":"ContainerDied","Data":"bdc76a595073711cbaab34eb78f16278aad8906cbde528f81ac024ee63d7dd10"} Sep 10 12:15:04.381032 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:04.380642 2583 scope.go:117] "RemoveContainer" containerID="a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7" Sep 10 12:15:04.389274 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:04.389258 2583 scope.go:117] "RemoveContainer" containerID="a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7" Sep 10 12:15:04.389583 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:15:04.389559 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7\": container with ID starting with a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7 not found: ID does not exist" containerID="a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7" Sep 10 12:15:04.389690 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:04.389585 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7"} err="failed to get container status \"a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7\": rpc error: code = NotFound desc = could not find container \"a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7\": container with ID starting with a2393f5de72298a7dd72a4447deb33d95f8cddbfc5df5680b858e7a4020285d7 not found: ID does not exist" Sep 10 12:15:04.399683 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:04.399656 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sxd2v"] Sep 10 12:15:04.406910 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:04.406887 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sxd2v"] Sep 10 12:15:05.902132 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:05.902100 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0178d3e-ece3-438b-83f5-bbe2da743aa0" path="/var/lib/kubelet/pods/a0178d3e-ece3-438b-83f5-bbe2da743aa0/volumes" Sep 10 12:15:24.360772 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.360732 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ts68j"] Sep 10 12:15:24.361294 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.361102 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0178d3e-ece3-438b-83f5-bbe2da743aa0" containerName="authorino" Sep 10 12:15:24.361294 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.361113 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0178d3e-ece3-438b-83f5-bbe2da743aa0" containerName="authorino" Sep 10 12:15:24.361294 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.361168 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0178d3e-ece3-438b-83f5-bbe2da743aa0" containerName="authorino" Sep 10 12:15:24.366413 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.366381 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:24.369593 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.369570 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 12:15:24.369861 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.369839 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 12:15:24.369945 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.369861 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 10 12:15:24.370320 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.370302 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zl4cx\"" Sep 10 12:15:24.384140 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.384117 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ts68j"] Sep 10 12:15:24.485418 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.485376 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/91ad704e-0dd3-4ca3-9f56-6c77d50c5a86-data\") pod \"postgres-69fb4594cf-ts68j\" (UID: \"91ad704e-0dd3-4ca3-9f56-6c77d50c5a86\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:24.485588 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.485464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7425k\" (UniqueName: \"kubernetes.io/projected/91ad704e-0dd3-4ca3-9f56-6c77d50c5a86-kube-api-access-7425k\") pod \"postgres-69fb4594cf-ts68j\" (UID: \"91ad704e-0dd3-4ca3-9f56-6c77d50c5a86\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:24.586434 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.586400 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7425k\" (UniqueName: \"kubernetes.io/projected/91ad704e-0dd3-4ca3-9f56-6c77d50c5a86-kube-api-access-7425k\") pod \"postgres-69fb4594cf-ts68j\" (UID: \"91ad704e-0dd3-4ca3-9f56-6c77d50c5a86\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:24.586607 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.586466 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/91ad704e-0dd3-4ca3-9f56-6c77d50c5a86-data\") pod \"postgres-69fb4594cf-ts68j\" (UID: \"91ad704e-0dd3-4ca3-9f56-6c77d50c5a86\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:24.586783 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.586765 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/91ad704e-0dd3-4ca3-9f56-6c77d50c5a86-data\") pod \"postgres-69fb4594cf-ts68j\" (UID: \"91ad704e-0dd3-4ca3-9f56-6c77d50c5a86\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:24.603549 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.603514 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7425k\" (UniqueName: \"kubernetes.io/projected/91ad704e-0dd3-4ca3-9f56-6c77d50c5a86-kube-api-access-7425k\") pod \"postgres-69fb4594cf-ts68j\" (UID: \"91ad704e-0dd3-4ca3-9f56-6c77d50c5a86\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:24.678661 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.678562 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:24.950950 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:24.950924 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ts68j"] Sep 10 12:15:24.953442 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:15:24.953415 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ad704e_0dd3_4ca3_9f56_6c77d50c5a86.slice/crio-6e0579d14ca3175544c40278cb30d9475e1ecbb4af802cd6a086f45b5591aa01 WatchSource:0}: Error finding container 6e0579d14ca3175544c40278cb30d9475e1ecbb4af802cd6a086f45b5591aa01: Status 404 returned error can't find the container with id 6e0579d14ca3175544c40278cb30d9475e1ecbb4af802cd6a086f45b5591aa01 Sep 10 12:15:25.486962 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:25.486927 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" event={"ID":"91ad704e-0dd3-4ca3-9f56-6c77d50c5a86","Type":"ContainerStarted","Data":"6e0579d14ca3175544c40278cb30d9475e1ecbb4af802cd6a086f45b5591aa01"} Sep 10 12:15:30.514792 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:30.514757 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" event={"ID":"91ad704e-0dd3-4ca3-9f56-6c77d50c5a86","Type":"ContainerStarted","Data":"64d9b8848568a335c6076f7d76442cef80778322301dba30e5ef105e0894cd92"} Sep 10 12:15:30.515219 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:30.514928 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:30.540847 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:30.540795 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" podStartSLOduration=1.822291321 podStartE2EDuration="6.540775637s" totalImagesPullingTime="4.718484316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:30.539599381 +0000 UTC m=+551.116922992" watchObservedRunningTime="2026-09-10 12:15:30.540775637 +0000 UTC m=+551.118099246" Sep 10 12:15:36.548829 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:36.548799 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ts68j" Sep 10 12:15:37.165874 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.165830 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-rbw4f"] Sep 10 12:15:37.170476 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.170455 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-rbw4f" Sep 10 12:15:37.172944 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.172923 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-glkkw\"" Sep 10 12:15:37.176775 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.176747 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-rbw4f"] Sep 10 12:15:37.258472 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.258434 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-rbw4f"] Sep 10 12:15:37.258731 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:15:37.258710 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-kznn8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-rbw4f" podUID="88846813-8894-4d9a-89ff-af3b1bdd6030" Sep 10 12:15:37.310166 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.310124 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kznn8\" (UniqueName: \"kubernetes.io/projected/88846813-8894-4d9a-89ff-af3b1bdd6030-kube-api-access-kznn8\") pod \"authorino-557f876f69-rbw4f\" (UID: \"88846813-8894-4d9a-89ff-af3b1bdd6030\") " pod="kuadrant-system/authorino-557f876f69-rbw4f" Sep 10 12:15:37.411576 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.411543 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kznn8\" (UniqueName: \"kubernetes.io/projected/88846813-8894-4d9a-89ff-af3b1bdd6030-kube-api-access-kznn8\") pod \"authorino-557f876f69-rbw4f\" (UID: \"88846813-8894-4d9a-89ff-af3b1bdd6030\") " pod="kuadrant-system/authorino-557f876f69-rbw4f" Sep 10 12:15:37.422495 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.422425 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kznn8\" (UniqueName: \"kubernetes.io/projected/88846813-8894-4d9a-89ff-af3b1bdd6030-kube-api-access-kznn8\") pod \"authorino-557f876f69-rbw4f\" (UID: \"88846813-8894-4d9a-89ff-af3b1bdd6030\") " pod="kuadrant-system/authorino-557f876f69-rbw4f" Sep 10 12:15:37.545160 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.545132 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-rbw4f" Sep 10 12:15:37.552851 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.552824 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-rbw4f" Sep 10 12:15:37.714964 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.714871 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kznn8\" (UniqueName: \"kubernetes.io/projected/88846813-8894-4d9a-89ff-af3b1bdd6030-kube-api-access-kznn8\") pod \"88846813-8894-4d9a-89ff-af3b1bdd6030\" (UID: \"88846813-8894-4d9a-89ff-af3b1bdd6030\") " Sep 10 12:15:37.717025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.716993 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88846813-8894-4d9a-89ff-af3b1bdd6030-kube-api-access-kznn8" (OuterVolumeSpecName: "kube-api-access-kznn8") pod "88846813-8894-4d9a-89ff-af3b1bdd6030" (UID: "88846813-8894-4d9a-89ff-af3b1bdd6030"). InnerVolumeSpecName "kube-api-access-kznn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:15:37.815885 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:37.815847 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kznn8\" (UniqueName: \"kubernetes.io/projected/88846813-8894-4d9a-89ff-af3b1bdd6030-kube-api-access-kznn8\") on node \"ip-10-0-141-9.ec2.internal\" DevicePath \"\"" Sep 10 12:15:38.549601 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:38.549571 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-rbw4f" Sep 10 12:15:38.584547 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:38.584518 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-rbw4f"] Sep 10 12:15:38.593342 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:38.593313 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-rbw4f"] Sep 10 12:15:39.902758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:15:39.902724 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88846813-8894-4d9a-89ff-af3b1bdd6030" path="/var/lib/kubelet/pods/88846813-8894-4d9a-89ff-af3b1bdd6030/volumes" Sep 10 12:16:49.647693 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.647612 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7"] Sep 10 12:16:49.650970 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.650955 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.659135 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.659111 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 12:16:49.659253 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.659111 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-82bff\"" Sep 10 12:16:49.659253 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.659148 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 12:16:49.659493 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.659478 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 10 12:16:49.673539 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.673516 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7"] Sep 10 12:16:49.759148 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.759108 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8f7a596-0d80-499a-b8ef-4a284d8fb973-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.759346 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.759191 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.759346 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.759224 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.759346 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.759290 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.759564 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.759410 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t9hv\" (UniqueName: \"kubernetes.io/projected/b8f7a596-0d80-499a-b8ef-4a284d8fb973-kube-api-access-6t9hv\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.759564 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.759516 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.859968 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.859925 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.859968 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.859974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t9hv\" (UniqueName: \"kubernetes.io/projected/b8f7a596-0d80-499a-b8ef-4a284d8fb973-kube-api-access-6t9hv\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.860232 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.860026 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.860232 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.860046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8f7a596-0d80-499a-b8ef-4a284d8fb973-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.860232 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.860062 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.860232 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.860088 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.860509 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.860485 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.860565 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.860530 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.860565 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.860547 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.862611 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.862587 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8f7a596-0d80-499a-b8ef-4a284d8fb973-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.862777 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.862761 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8f7a596-0d80-499a-b8ef-4a284d8fb973-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.872109 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.872085 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t9hv\" (UniqueName: \"kubernetes.io/projected/b8f7a596-0d80-499a-b8ef-4a284d8fb973-kube-api-access-6t9hv\") pod \"e2e-distinct-2-simulated-kserve-859856b874-x6ff7\" (UID: \"b8f7a596-0d80-499a-b8ef-4a284d8fb973\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:49.960937 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:49.960853 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:50.266508 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:50.266291 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7"] Sep 10 12:16:50.269276 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:16:50.269232 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f7a596_0d80_499a_b8ef_4a284d8fb973.slice/crio-5e9c2f959e231ba5467b978f8c7027af0430d68f81aa5fae68e6f28a94c401db WatchSource:0}: Error finding container 5e9c2f959e231ba5467b978f8c7027af0430d68f81aa5fae68e6f28a94c401db: Status 404 returned error can't find the container with id 5e9c2f959e231ba5467b978f8c7027af0430d68f81aa5fae68e6f28a94c401db Sep 10 12:16:50.270605 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:50.270576 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:16:50.889799 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:50.889747 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" event={"ID":"b8f7a596-0d80-499a-b8ef-4a284d8fb973","Type":"ContainerStarted","Data":"5e9c2f959e231ba5467b978f8c7027af0430d68f81aa5fae68e6f28a94c401db"} Sep 10 12:16:51.895866 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:51.895786 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" containerID="3d487d879ef700801eeda6f6cd7a403112e79b2d4175c041cb2edadc5e0b432d" exitCode=0 Sep 10 12:16:51.895866 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:51.895822 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" event={"ID":"b8f7a596-0d80-499a-b8ef-4a284d8fb973","Type":"ContainerDied","Data":"3d487d879ef700801eeda6f6cd7a403112e79b2d4175c041cb2edadc5e0b432d"} Sep 10 12:16:51.896252 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:51.896217 2583 scope.go:117] "RemoveContainer" containerID="3d487d879ef700801eeda6f6cd7a403112e79b2d4175c041cb2edadc5e0b432d" Sep 10 12:16:52.903042 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:52.903007 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" containerID="be14c6b77d30153b9caecd3f0bcbb57247bd77a597ddf49a6fbce2cc10b2a727" exitCode=0 Sep 10 12:16:52.903520 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:52.903096 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" event={"ID":"b8f7a596-0d80-499a-b8ef-4a284d8fb973","Type":"ContainerDied","Data":"be14c6b77d30153b9caecd3f0bcbb57247bd77a597ddf49a6fbce2cc10b2a727"} Sep 10 12:16:52.903520 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:52.903153 2583 scope.go:117] "RemoveContainer" containerID="3d487d879ef700801eeda6f6cd7a403112e79b2d4175c041cb2edadc5e0b432d" Sep 10 12:16:52.903520 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:52.903437 2583 scope.go:117] "RemoveContainer" containerID="be14c6b77d30153b9caecd3f0bcbb57247bd77a597ddf49a6fbce2cc10b2a727" Sep 10 12:16:52.903652 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:16:52.903635 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:16:53.908922 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:53.908895 2583 scope.go:117] "RemoveContainer" containerID="be14c6b77d30153b9caecd3f0bcbb57247bd77a597ddf49a6fbce2cc10b2a727" Sep 10 12:16:53.909293 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:16:53.909063 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:16:59.961169 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:59.961137 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:59.961577 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:59.961226 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:59.961577 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:59.961279 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:59.961577 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:59.961301 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:16:59.961757 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:16:59.961743 2583 scope.go:117] "RemoveContainer" containerID="be14c6b77d30153b9caecd3f0bcbb57247bd77a597ddf49a6fbce2cc10b2a727" Sep 10 12:16:59.961974 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:16:59.961953 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:17:00.948747 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:00.948711 2583 scope.go:117] "RemoveContainer" containerID="be14c6b77d30153b9caecd3f0bcbb57247bd77a597ddf49a6fbce2cc10b2a727" Sep 10 12:17:00.948952 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:00.948929 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:17:14.909711 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.909672 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl"] Sep 10 12:17:14.915636 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.915605 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:14.918059 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.918037 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 10 12:17:14.927417 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.927391 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl"] Sep 10 12:17:14.989205 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.989154 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:14.989464 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.989230 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:14.989464 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.989262 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:14.989464 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.989282 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcvwk\" (UniqueName: \"kubernetes.io/projected/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-kube-api-access-rcvwk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:14.989464 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.989329 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:14.989464 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:14.989344 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.090841 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.090803 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.090841 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.090840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.091106 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.090859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.091106 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.090876 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcvwk\" (UniqueName: \"kubernetes.io/projected/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-kube-api-access-rcvwk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.091106 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.090916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.091106 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.090938 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.091317 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.091238 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.091317 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.091287 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.091447 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.091317 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.093206 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.093184 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.093366 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.093329 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.100281 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.100252 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcvwk\" (UniqueName: \"kubernetes.io/projected/efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1-kube-api-access-rcvwk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5ckbl\" (UID: \"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.227587 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.227493 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:15.517081 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.517004 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl"] Sep 10 12:17:15.520082 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:17:15.520053 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc0d1d8_dc9b_4c65_93a6_e634a2cf43f1.slice/crio-b37caf444b65cf49f1d669d03d38e840446be42a096bdc6c3f2ab98cfa8db473 WatchSource:0}: Error finding container b37caf444b65cf49f1d669d03d38e840446be42a096bdc6c3f2ab98cfa8db473: Status 404 returned error can't find the container with id b37caf444b65cf49f1d669d03d38e840446be42a096bdc6c3f2ab98cfa8db473 Sep 10 12:17:15.896713 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:15.896683 2583 scope.go:117] "RemoveContainer" containerID="be14c6b77d30153b9caecd3f0bcbb57247bd77a597ddf49a6fbce2cc10b2a727" Sep 10 12:17:16.019444 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:16.019408 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" event={"ID":"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1","Type":"ContainerStarted","Data":"96419d0ac009fe80592d68aeba4b89eb3a26423c363dc8ce2542e32935249847"} Sep 10 12:17:16.019854 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:16.019462 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" event={"ID":"efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1","Type":"ContainerStarted","Data":"b37caf444b65cf49f1d669d03d38e840446be42a096bdc6c3f2ab98cfa8db473"} Sep 10 12:17:16.045580 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:16.045526 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" podStartSLOduration=1.852647744 podStartE2EDuration="2.045508828s" totalImagesPullingTime="192.861084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:16.043706909 +0000 UTC m=+656.621030517" watchObservedRunningTime="2026-09-10 12:17:16.045508828 +0000 UTC m=+656.622832434" Sep 10 12:17:17.026683 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:17.026651 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" containerID="cb8c58588686e83e30ec1b4f2da508aae4f1788272a3a0602328387619ce8902" exitCode=0 Sep 10 12:17:17.027082 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:17.026726 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" event={"ID":"b8f7a596-0d80-499a-b8ef-4a284d8fb973","Type":"ContainerDied","Data":"cb8c58588686e83e30ec1b4f2da508aae4f1788272a3a0602328387619ce8902"} Sep 10 12:17:17.027082 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:17.026785 2583 scope.go:117] "RemoveContainer" containerID="be14c6b77d30153b9caecd3f0bcbb57247bd77a597ddf49a6fbce2cc10b2a727" Sep 10 12:17:17.027250 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:17.027234 2583 scope.go:117] "RemoveContainer" containerID="cb8c58588686e83e30ec1b4f2da508aae4f1788272a3a0602328387619ce8902" Sep 10 12:17:17.027459 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:17.027440 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:17:19.961462 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:19.961430 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:17:19.961865 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:19.961488 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:17:19.961865 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:19.961506 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:17:19.961865 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:19.961518 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:17:19.961865 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:19.961862 2583 scope.go:117] "RemoveContainer" containerID="cb8c58588686e83e30ec1b4f2da508aae4f1788272a3a0602328387619ce8902" Sep 10 12:17:19.962056 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:19.962036 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:17:25.228615 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:25.228573 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:25.229108 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:25.228645 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:25.241276 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:25.241249 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:26.082037 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:26.081999 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5ckbl" Sep 10 12:17:32.896328 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:32.896288 2583 scope.go:117] "RemoveContainer" containerID="cb8c58588686e83e30ec1b4f2da508aae4f1788272a3a0602328387619ce8902" Sep 10 12:17:32.896736 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:32.896528 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:17:34.733025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.732991 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc"] Sep 10 12:17:34.740798 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.740774 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.743270 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.743246 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 10 12:17:34.751109 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.751083 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc"] Sep 10 12:17:34.756172 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.756143 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.756308 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.756183 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-887gm\" (UniqueName: \"kubernetes.io/projected/185b3e98-9ae1-488c-9359-0142e755492b-kube-api-access-887gm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.756308 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.756244 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185b3e98-9ae1-488c-9359-0142e755492b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.756488 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.756321 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.756488 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.756361 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.756488 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.756393 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.856885 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.856853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185b3e98-9ae1-488c-9359-0142e755492b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.857079 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.856958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.857079 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.856989 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.857079 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.857010 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.857079 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.857058 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.857274 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.857097 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-887gm\" (UniqueName: \"kubernetes.io/projected/185b3e98-9ae1-488c-9359-0142e755492b-kube-api-access-887gm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.857507 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.857479 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.857652 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.857589 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.857652 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.857599 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.859643 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.859611 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/185b3e98-9ae1-488c-9359-0142e755492b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.859894 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.859877 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185b3e98-9ae1-488c-9359-0142e755492b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:34.866098 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:34.866078 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-887gm\" (UniqueName: \"kubernetes.io/projected/185b3e98-9ae1-488c-9359-0142e755492b-kube-api-access-887gm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc\" (UID: \"185b3e98-9ae1-488c-9359-0142e755492b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:35.052309 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:35.052275 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:35.337084 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:35.336126 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc"] Sep 10 12:17:35.341688 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:17:35.341646 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185b3e98_9ae1_488c_9359_0142e755492b.slice/crio-575319695db542cdd70865fc5819a3d719aee0ec598401a3bb36efd79d540447 WatchSource:0}: Error finding container 575319695db542cdd70865fc5819a3d719aee0ec598401a3bb36efd79d540447: Status 404 returned error can't find the container with id 575319695db542cdd70865fc5819a3d719aee0ec598401a3bb36efd79d540447 Sep 10 12:17:36.121113 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:36.121077 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" event={"ID":"185b3e98-9ae1-488c-9359-0142e755492b","Type":"ContainerStarted","Data":"d93e3db782f9d42601b6cd7490e8923ab6f0d6efdac9e52826887300428f78b4"} Sep 10 12:17:36.121538 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:36.121140 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" event={"ID":"185b3e98-9ae1-488c-9359-0142e755492b","Type":"ContainerStarted","Data":"575319695db542cdd70865fc5819a3d719aee0ec598401a3bb36efd79d540447"} Sep 10 12:17:36.153603 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:36.153546 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" podStartSLOduration=1.978861659 podStartE2EDuration="2.153528888s" totalImagesPullingTime="174.667229ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:36.151790192 +0000 UTC m=+676.729113802" watchObservedRunningTime="2026-09-10 12:17:36.153528888 +0000 UTC m=+676.730852497" Sep 10 12:17:43.530595 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.530553 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq"] Sep 10 12:17:43.534190 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.534167 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.536717 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.536696 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 12:17:43.544923 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.544893 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq"] Sep 10 12:17:43.639789 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.639748 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.639789 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.639789 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.640017 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.639877 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.640017 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.639914 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be8dde9d-0117-44f1-b835-2859431ab693-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.640017 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.639934 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.640017 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.639969 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j49g\" (UniqueName: \"kubernetes.io/projected/be8dde9d-0117-44f1-b835-2859431ab693-kube-api-access-2j49g\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.741393 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.741340 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.741393 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.741392 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.741608 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.741433 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.741608 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.741453 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be8dde9d-0117-44f1-b835-2859431ab693-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.741608 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.741471 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.741608 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.741494 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j49g\" (UniqueName: \"kubernetes.io/projected/be8dde9d-0117-44f1-b835-2859431ab693-kube-api-access-2j49g\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.741889 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.741857 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.741889 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.741877 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.742045 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.741925 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.743836 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.743814 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be8dde9d-0117-44f1-b835-2859431ab693-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.744105 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.744086 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be8dde9d-0117-44f1-b835-2859431ab693-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.750367 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.750328 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j49g\" (UniqueName: \"kubernetes.io/projected/be8dde9d-0117-44f1-b835-2859431ab693-kube-api-access-2j49g\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq\" (UID: \"be8dde9d-0117-44f1-b835-2859431ab693\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:43.848461 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:43.848423 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:44.130464 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:17:44.130431 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8dde9d_0117_44f1_b835_2859431ab693.slice/crio-137a4fd7d8959f81d791118edb71b5b1a4799364d305eebb1aa8b78440306e9d WatchSource:0}: Error finding container 137a4fd7d8959f81d791118edb71b5b1a4799364d305eebb1aa8b78440306e9d: Status 404 returned error can't find the container with id 137a4fd7d8959f81d791118edb71b5b1a4799364d305eebb1aa8b78440306e9d Sep 10 12:17:44.132395 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:44.132369 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq"] Sep 10 12:17:44.161465 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:44.161431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" event={"ID":"be8dde9d-0117-44f1-b835-2859431ab693","Type":"ContainerStarted","Data":"137a4fd7d8959f81d791118edb71b5b1a4799364d305eebb1aa8b78440306e9d"} Sep 10 12:17:44.896948 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:44.896918 2583 scope.go:117] "RemoveContainer" containerID="cb8c58588686e83e30ec1b4f2da508aae4f1788272a3a0602328387619ce8902" Sep 10 12:17:45.052819 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:45.052781 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:45.053010 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:45.052844 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:45.065690 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:45.065653 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:45.169141 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:45.169114 2583 generic.go:356] "Generic (PLEG): container finished" podID="be8dde9d-0117-44f1-b835-2859431ab693" containerID="7baeb0714ea7055c8b16bebfc63e6e7d31b8ce327207435c39c0f93e9259f6db" exitCode=0 Sep 10 12:17:45.169283 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:45.169189 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" event={"ID":"be8dde9d-0117-44f1-b835-2859431ab693","Type":"ContainerDied","Data":"7baeb0714ea7055c8b16bebfc63e6e7d31b8ce327207435c39c0f93e9259f6db"} Sep 10 12:17:45.169576 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:45.169551 2583 scope.go:117] "RemoveContainer" containerID="7baeb0714ea7055c8b16bebfc63e6e7d31b8ce327207435c39c0f93e9259f6db" Sep 10 12:17:45.171306 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:45.171283 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" event={"ID":"b8f7a596-0d80-499a-b8ef-4a284d8fb973","Type":"ContainerStarted","Data":"bbf3516938917a7ec4262f4d5a238c9c0435466c80b30b50b6c42c71ce7713f2"} Sep 10 12:17:45.183608 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:45.183578 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc" Sep 10 12:17:45.243009 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:45.242945 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podStartSLOduration=54.298508073 podStartE2EDuration="56.242922396s" totalImagesPullingTime="1.944414323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:16:49 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:45.241032957 +0000 UTC m=+685.818356583" watchObservedRunningTime="2026-09-10 12:17:45.242922396 +0000 UTC m=+685.820246006" Sep 10 12:17:46.179019 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:46.178987 2583 generic.go:356] "Generic (PLEG): container finished" podID="be8dde9d-0117-44f1-b835-2859431ab693" containerID="e77a707b5f113df28eb8fd3625ee024c3d45f98d85410b8d33d371db895ecbd9" exitCode=0 Sep 10 12:17:46.179621 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:46.179082 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" event={"ID":"be8dde9d-0117-44f1-b835-2859431ab693","Type":"ContainerDied","Data":"e77a707b5f113df28eb8fd3625ee024c3d45f98d85410b8d33d371db895ecbd9"} Sep 10 12:17:46.179621 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:46.179153 2583 scope.go:117] "RemoveContainer" containerID="7baeb0714ea7055c8b16bebfc63e6e7d31b8ce327207435c39c0f93e9259f6db" Sep 10 12:17:46.179621 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:46.179436 2583 scope.go:117] "RemoveContainer" containerID="e77a707b5f113df28eb8fd3625ee024c3d45f98d85410b8d33d371db895ecbd9" Sep 10 12:17:46.179801 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:46.179675 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:17:46.181291 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:46.181270 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" containerID="bbf3516938917a7ec4262f4d5a238c9c0435466c80b30b50b6c42c71ce7713f2" exitCode=0 Sep 10 12:17:46.181419 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:46.181376 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" event={"ID":"b8f7a596-0d80-499a-b8ef-4a284d8fb973","Type":"ContainerDied","Data":"bbf3516938917a7ec4262f4d5a238c9c0435466c80b30b50b6c42c71ce7713f2"} Sep 10 12:17:46.181840 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:46.181821 2583 scope.go:117] "RemoveContainer" containerID="bbf3516938917a7ec4262f4d5a238c9c0435466c80b30b50b6c42c71ce7713f2" Sep 10 12:17:46.182005 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:46.181984 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:17:46.191135 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:46.191114 2583 scope.go:117] "RemoveContainer" containerID="cb8c58588686e83e30ec1b4f2da508aae4f1788272a3a0602328387619ce8902" Sep 10 12:17:47.189880 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:47.189848 2583 scope.go:117] "RemoveContainer" containerID="e77a707b5f113df28eb8fd3625ee024c3d45f98d85410b8d33d371db895ecbd9" Sep 10 12:17:47.190378 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:47.190094 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:17:49.961542 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:49.961497 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:17:49.961979 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:49.961578 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:17:49.961979 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:49.961602 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:17:49.961979 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:49.961624 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:17:49.962080 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:49.962040 2583 scope.go:117] "RemoveContainer" containerID="bbf3516938917a7ec4262f4d5a238c9c0435466c80b30b50b6c42c71ce7713f2" Sep 10 12:17:49.962270 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:49.962249 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:17:53.849544 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:53.849505 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:53.849949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:53.849566 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:53.849949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:53.849582 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:53.849949 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:53.849595 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:17:53.850110 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:53.850086 2583 scope.go:117] "RemoveContainer" containerID="e77a707b5f113df28eb8fd3625ee024c3d45f98d85410b8d33d371db895ecbd9" Sep 10 12:17:53.850319 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:53.850299 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:17:54.224589 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:54.224507 2583 scope.go:117] "RemoveContainer" containerID="e77a707b5f113df28eb8fd3625ee024c3d45f98d85410b8d33d371db895ecbd9" Sep 10 12:17:54.224731 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:17:54.224690 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:17:59.128107 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.128071 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw"] Sep 10 12:17:59.133118 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.133092 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.135934 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.135911 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 10 12:17:59.142956 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.142929 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw"] Sep 10 12:17:59.292801 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.292756 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntdzd\" (UniqueName: \"kubernetes.io/projected/3a339f96-7bfd-4023-9926-e4b60b5118f1-kube-api-access-ntdzd\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.292801 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.292804 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.293045 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.292872 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.293045 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.292924 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.293045 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.292959 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.293045 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.292977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a339f96-7bfd-4023-9926-e4b60b5118f1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.393910 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.393816 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.393910 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.393860 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.393910 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.393886 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.393910 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.393905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a339f96-7bfd-4023-9926-e4b60b5118f1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.394250 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.394014 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntdzd\" (UniqueName: \"kubernetes.io/projected/3a339f96-7bfd-4023-9926-e4b60b5118f1-kube-api-access-ntdzd\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.394250 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.394040 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.394250 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.394226 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.394426 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.394303 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.394426 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.394315 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.396537 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.396518 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a339f96-7bfd-4023-9926-e4b60b5118f1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.396652 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.396634 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a339f96-7bfd-4023-9926-e4b60b5118f1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.405899 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.405873 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntdzd\" (UniqueName: \"kubernetes.io/projected/3a339f96-7bfd-4023-9926-e4b60b5118f1-kube-api-access-ntdzd\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-t79nw\" (UID: \"3a339f96-7bfd-4023-9926-e4b60b5118f1\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.442985 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.442947 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:17:59.727032 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:17:59.727007 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw"] Sep 10 12:17:59.728884 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:17:59.728859 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a339f96_7bfd_4023_9926_e4b60b5118f1.slice/crio-2b1b9ff6ad7e27f521c06ab844fba4b5897d2811487954541ab5a1e6a5cb172d WatchSource:0}: Error finding container 2b1b9ff6ad7e27f521c06ab844fba4b5897d2811487954541ab5a1e6a5cb172d: Status 404 returned error can't find the container with id 2b1b9ff6ad7e27f521c06ab844fba4b5897d2811487954541ab5a1e6a5cb172d Sep 10 12:18:00.259304 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:00.259200 2583 generic.go:356] "Generic (PLEG): container finished" podID="3a339f96-7bfd-4023-9926-e4b60b5118f1" containerID="81f9b2d0f5a55584130f7f5df3ab5dbf6cf4c29cb48271201c2fb834eb4538c4" exitCode=0 Sep 10 12:18:00.259304 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:00.259294 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" event={"ID":"3a339f96-7bfd-4023-9926-e4b60b5118f1","Type":"ContainerDied","Data":"81f9b2d0f5a55584130f7f5df3ab5dbf6cf4c29cb48271201c2fb834eb4538c4"} Sep 10 12:18:00.259759 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:00.259331 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" event={"ID":"3a339f96-7bfd-4023-9926-e4b60b5118f1","Type":"ContainerStarted","Data":"2b1b9ff6ad7e27f521c06ab844fba4b5897d2811487954541ab5a1e6a5cb172d"} Sep 10 12:18:00.259759 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:00.259591 2583 scope.go:117] "RemoveContainer" containerID="81f9b2d0f5a55584130f7f5df3ab5dbf6cf4c29cb48271201c2fb834eb4538c4" Sep 10 12:18:00.896588 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:00.896553 2583 scope.go:117] "RemoveContainer" containerID="bbf3516938917a7ec4262f4d5a238c9c0435466c80b30b50b6c42c71ce7713f2" Sep 10 12:18:00.896776 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:00.896741 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:18:01.268040 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:01.267953 2583 generic.go:356] "Generic (PLEG): container finished" podID="3a339f96-7bfd-4023-9926-e4b60b5118f1" containerID="9a5a91422d684b4ba8fd32cb8548f7888eb5b506f7ddcb82ead2762e394e2bd1" exitCode=0 Sep 10 12:18:01.268528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:01.268032 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" event={"ID":"3a339f96-7bfd-4023-9926-e4b60b5118f1","Type":"ContainerDied","Data":"9a5a91422d684b4ba8fd32cb8548f7888eb5b506f7ddcb82ead2762e394e2bd1"} Sep 10 12:18:01.268528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:01.268079 2583 scope.go:117] "RemoveContainer" containerID="81f9b2d0f5a55584130f7f5df3ab5dbf6cf4c29cb48271201c2fb834eb4538c4" Sep 10 12:18:01.268528 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:01.268396 2583 scope.go:117] "RemoveContainer" containerID="9a5a91422d684b4ba8fd32cb8548f7888eb5b506f7ddcb82ead2762e394e2bd1" Sep 10 12:18:01.268673 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:01.268640 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:18:02.275386 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:02.275339 2583 scope.go:117] "RemoveContainer" containerID="9a5a91422d684b4ba8fd32cb8548f7888eb5b506f7ddcb82ead2762e394e2bd1" Sep 10 12:18:02.275776 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:02.275536 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:18:08.896885 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:08.896848 2583 scope.go:117] "RemoveContainer" containerID="e77a707b5f113df28eb8fd3625ee024c3d45f98d85410b8d33d371db895ecbd9" Sep 10 12:18:09.309164 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:09.309132 2583 generic.go:356] "Generic (PLEG): container finished" podID="be8dde9d-0117-44f1-b835-2859431ab693" containerID="459ae8c7e541a0fda2668ff18bf9de5f5cc8ba513440ba25f3622778a4550d44" exitCode=0 Sep 10 12:18:09.309345 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:09.309210 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" event={"ID":"be8dde9d-0117-44f1-b835-2859431ab693","Type":"ContainerDied","Data":"459ae8c7e541a0fda2668ff18bf9de5f5cc8ba513440ba25f3622778a4550d44"} Sep 10 12:18:09.309345 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:09.309274 2583 scope.go:117] "RemoveContainer" containerID="e77a707b5f113df28eb8fd3625ee024c3d45f98d85410b8d33d371db895ecbd9" Sep 10 12:18:09.309709 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:09.309693 2583 scope.go:117] "RemoveContainer" containerID="459ae8c7e541a0fda2668ff18bf9de5f5cc8ba513440ba25f3622778a4550d44" Sep 10 12:18:09.309971 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:09.309941 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:18:09.444118 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:09.444022 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:09.444118 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:09.444069 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:09.444118 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:09.444079 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:09.444118 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:09.444088 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:09.444532 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:09.444517 2583 scope.go:117] "RemoveContainer" containerID="9a5a91422d684b4ba8fd32cb8548f7888eb5b506f7ddcb82ead2762e394e2bd1" Sep 10 12:18:09.444704 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:09.444686 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:18:12.896760 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:12.896680 2583 scope.go:117] "RemoveContainer" containerID="bbf3516938917a7ec4262f4d5a238c9c0435466c80b30b50b6c42c71ce7713f2" Sep 10 12:18:12.897111 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:12.896865 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:18:13.849133 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:13.849095 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:18:13.849325 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:13.849161 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:18:13.849325 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:13.849177 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:18:13.849325 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:13.849194 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:18:13.849588 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:13.849573 2583 scope.go:117] "RemoveContainer" containerID="459ae8c7e541a0fda2668ff18bf9de5f5cc8ba513440ba25f3622778a4550d44" Sep 10 12:18:13.849754 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:13.849736 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:18:24.896423 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:24.896380 2583 scope.go:117] "RemoveContainer" containerID="9a5a91422d684b4ba8fd32cb8548f7888eb5b506f7ddcb82ead2762e394e2bd1" Sep 10 12:18:25.393023 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:25.392995 2583 generic.go:356] "Generic (PLEG): container finished" podID="3a339f96-7bfd-4023-9926-e4b60b5118f1" containerID="7e9212dfb05036df4af4f966985f7df679d1a9c7d10d301e050f5c1db06157eb" exitCode=0 Sep 10 12:18:25.393198 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:25.393072 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" event={"ID":"3a339f96-7bfd-4023-9926-e4b60b5118f1","Type":"ContainerDied","Data":"7e9212dfb05036df4af4f966985f7df679d1a9c7d10d301e050f5c1db06157eb"} Sep 10 12:18:25.393198 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:25.393126 2583 scope.go:117] "RemoveContainer" containerID="9a5a91422d684b4ba8fd32cb8548f7888eb5b506f7ddcb82ead2762e394e2bd1" Sep 10 12:18:25.393586 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:25.393558 2583 scope.go:117] "RemoveContainer" containerID="7e9212dfb05036df4af4f966985f7df679d1a9c7d10d301e050f5c1db06157eb" Sep 10 12:18:25.393791 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:25.393769 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:18:27.896828 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:27.896796 2583 scope.go:117] "RemoveContainer" containerID="bbf3516938917a7ec4262f4d5a238c9c0435466c80b30b50b6c42c71ce7713f2" Sep 10 12:18:28.411318 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:28.411234 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" exitCode=0 Sep 10 12:18:28.411318 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:28.411279 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" event={"ID":"b8f7a596-0d80-499a-b8ef-4a284d8fb973","Type":"ContainerDied","Data":"bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744"} Sep 10 12:18:28.411526 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:28.411399 2583 scope.go:117] "RemoveContainer" containerID="bbf3516938917a7ec4262f4d5a238c9c0435466c80b30b50b6c42c71ce7713f2" Sep 10 12:18:28.411809 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:28.411792 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:18:28.412039 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:28.412020 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:18:28.896516 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:28.896480 2583 scope.go:117] "RemoveContainer" containerID="459ae8c7e541a0fda2668ff18bf9de5f5cc8ba513440ba25f3622778a4550d44" Sep 10 12:18:28.896704 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:28.896672 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:18:29.443377 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.443311 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:29.443377 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.443389 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:29.443833 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.443403 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:29.443833 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.443411 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:29.443833 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.443796 2583 scope.go:117] "RemoveContainer" containerID="7e9212dfb05036df4af4f966985f7df679d1a9c7d10d301e050f5c1db06157eb" Sep 10 12:18:29.444011 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:29.443982 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:18:29.961611 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.961572 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:18:29.961796 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.961633 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:18:29.961796 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.961648 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:18:29.961796 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.961661 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:18:29.962045 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:29.962032 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:18:29.962223 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:29.962206 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:18:42.896671 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:42.896639 2583 scope.go:117] "RemoveContainer" containerID="459ae8c7e541a0fda2668ff18bf9de5f5cc8ba513440ba25f3622778a4550d44" Sep 10 12:18:42.897055 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:42.896727 2583 scope.go:117] "RemoveContainer" containerID="7e9212dfb05036df4af4f966985f7df679d1a9c7d10d301e050f5c1db06157eb" Sep 10 12:18:42.897055 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:42.896889 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:18:43.493329 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:43.493245 2583 generic.go:356] "Generic (PLEG): container finished" podID="be8dde9d-0117-44f1-b835-2859431ab693" containerID="7c2852ccfceac6134ac19053f544bc81c1c0951af209e7a258807764c156b5dc" exitCode=0 Sep 10 12:18:43.493329 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:43.493299 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" event={"ID":"be8dde9d-0117-44f1-b835-2859431ab693","Type":"ContainerDied","Data":"7c2852ccfceac6134ac19053f544bc81c1c0951af209e7a258807764c156b5dc"} Sep 10 12:18:43.493593 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:43.493370 2583 scope.go:117] "RemoveContainer" containerID="459ae8c7e541a0fda2668ff18bf9de5f5cc8ba513440ba25f3622778a4550d44" Sep 10 12:18:43.493753 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:43.493735 2583 scope.go:117] "RemoveContainer" containerID="7c2852ccfceac6134ac19053f544bc81c1c0951af209e7a258807764c156b5dc" Sep 10 12:18:43.493983 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:43.493961 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:18:43.848568 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:43.848533 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:18:43.848758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:43.848606 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:18:43.848758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:43.848653 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:18:43.848758 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:43.848668 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:18:43.896630 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:43.896603 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:18:43.896816 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:43.896795 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:18:44.503119 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:44.503090 2583 scope.go:117] "RemoveContainer" containerID="7c2852ccfceac6134ac19053f544bc81c1c0951af209e7a258807764c156b5dc" Sep 10 12:18:44.503289 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:44.503270 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:18:45.508583 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:45.508548 2583 scope.go:117] "RemoveContainer" containerID="7c2852ccfceac6134ac19053f544bc81c1c0951af209e7a258807764c156b5dc" Sep 10 12:18:45.509049 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:45.508790 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:18:55.896930 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:55.896899 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:18:55.897428 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:55.897102 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:18:57.896584 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:57.896545 2583 scope.go:117] "RemoveContainer" containerID="7e9212dfb05036df4af4f966985f7df679d1a9c7d10d301e050f5c1db06157eb" Sep 10 12:18:58.577662 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:58.577624 2583 generic.go:356] "Generic (PLEG): container finished" podID="3a339f96-7bfd-4023-9926-e4b60b5118f1" containerID="7d951c5bdd84b112ebab0e0c2ab082cf458d2d7cad3778e97205ec9ddf156218" exitCode=0 Sep 10 12:18:58.577868 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:58.577700 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" event={"ID":"3a339f96-7bfd-4023-9926-e4b60b5118f1","Type":"ContainerDied","Data":"7d951c5bdd84b112ebab0e0c2ab082cf458d2d7cad3778e97205ec9ddf156218"} Sep 10 12:18:58.577868 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:58.577750 2583 scope.go:117] "RemoveContainer" containerID="7e9212dfb05036df4af4f966985f7df679d1a9c7d10d301e050f5c1db06157eb" Sep 10 12:18:58.578168 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:58.578153 2583 scope.go:117] "RemoveContainer" containerID="7d951c5bdd84b112ebab0e0c2ab082cf458d2d7cad3778e97205ec9ddf156218" Sep 10 12:18:58.578389 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:58.578364 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:18:58.897141 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:58.897060 2583 scope.go:117] "RemoveContainer" containerID="7c2852ccfceac6134ac19053f544bc81c1c0951af209e7a258807764c156b5dc" Sep 10 12:18:58.897529 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:58.897238 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:18:59.443993 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:59.443938 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:59.444191 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:59.444013 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:59.444191 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:59.444025 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:59.444191 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:59.444034 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:18:59.584676 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:18:59.584650 2583 scope.go:117] "RemoveContainer" containerID="7d951c5bdd84b112ebab0e0c2ab082cf458d2d7cad3778e97205ec9ddf156218" Sep 10 12:18:59.584843 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:18:59.584823 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:19:08.897314 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:08.897282 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:19:08.897807 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:08.897507 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:19:10.896557 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:10.896523 2583 scope.go:117] "RemoveContainer" containerID="7c2852ccfceac6134ac19053f544bc81c1c0951af209e7a258807764c156b5dc" Sep 10 12:19:10.896939 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:10.896757 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:19:14.897010 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:14.896976 2583 scope.go:117] "RemoveContainer" containerID="7d951c5bdd84b112ebab0e0c2ab082cf458d2d7cad3778e97205ec9ddf156218" Sep 10 12:19:14.897530 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:14.897155 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:19:22.896525 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:22.896486 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:19:22.897013 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:22.896722 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:19:23.897273 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:23.897236 2583 scope.go:117] "RemoveContainer" containerID="7c2852ccfceac6134ac19053f544bc81c1c0951af209e7a258807764c156b5dc" Sep 10 12:19:24.717200 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:24.717169 2583 generic.go:356] "Generic (PLEG): container finished" podID="be8dde9d-0117-44f1-b835-2859431ab693" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" exitCode=0 Sep 10 12:19:24.717391 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:24.717244 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" event={"ID":"be8dde9d-0117-44f1-b835-2859431ab693","Type":"ContainerDied","Data":"21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38"} Sep 10 12:19:24.717391 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:24.717303 2583 scope.go:117] "RemoveContainer" containerID="7c2852ccfceac6134ac19053f544bc81c1c0951af209e7a258807764c156b5dc" Sep 10 12:19:24.717681 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:24.717660 2583 scope.go:117] "RemoveContainer" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" Sep 10 12:19:24.717887 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:24.717863 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:19:27.897040 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:27.897007 2583 scope.go:117] "RemoveContainer" containerID="7d951c5bdd84b112ebab0e0c2ab082cf458d2d7cad3778e97205ec9ddf156218" Sep 10 12:19:27.897579 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:27.897199 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:19:33.848771 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:33.848726 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:19:33.849244 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:33.848810 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:19:33.849244 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:33.848835 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:19:33.849244 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:33.848849 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:19:33.849244 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:33.849224 2583 scope.go:117] "RemoveContainer" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" Sep 10 12:19:33.849458 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:33.849437 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:19:33.897029 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:33.896988 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:19:33.897229 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:33.897166 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:19:39.899488 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:39.899454 2583 scope.go:117] "RemoveContainer" containerID="7d951c5bdd84b112ebab0e0c2ab082cf458d2d7cad3778e97205ec9ddf156218" Sep 10 12:19:40.799733 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:40.799647 2583 generic.go:356] "Generic (PLEG): container finished" podID="3a339f96-7bfd-4023-9926-e4b60b5118f1" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" exitCode=0 Sep 10 12:19:40.799893 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:40.799727 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" event={"ID":"3a339f96-7bfd-4023-9926-e4b60b5118f1","Type":"ContainerDied","Data":"7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab"} Sep 10 12:19:40.799893 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:40.799787 2583 scope.go:117] "RemoveContainer" containerID="7d951c5bdd84b112ebab0e0c2ab082cf458d2d7cad3778e97205ec9ddf156218" Sep 10 12:19:40.800180 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:40.800157 2583 scope.go:117] "RemoveContainer" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" Sep 10 12:19:40.800422 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:40.800403 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:19:47.896374 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:47.896328 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:19:47.896942 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:47.896499 2583 scope.go:117] "RemoveContainer" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" Sep 10 12:19:47.896942 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:47.896605 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:19:47.896942 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:47.896704 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:19:49.443919 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:49.443881 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:19:49.444306 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:49.443941 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:19:49.444306 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:49.443953 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:19:49.444306 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:49.443961 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:19:49.444580 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:19:49.444321 2583 scope.go:117] "RemoveContainer" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" Sep 10 12:19:49.444580 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:19:49.444505 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:20:02.896909 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:02.896873 2583 scope.go:117] "RemoveContainer" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" Sep 10 12:20:02.897339 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:02.897030 2583 scope.go:117] "RemoveContainer" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" Sep 10 12:20:02.897339 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:02.897074 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:20:02.897339 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:02.897161 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:20:02.897339 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:02.897169 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:20:03.914964 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:03.914930 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" exitCode=0 Sep 10 12:20:03.915468 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:03.915006 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" event={"ID":"b8f7a596-0d80-499a-b8ef-4a284d8fb973","Type":"ContainerDied","Data":"b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123"} Sep 10 12:20:03.915468 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:03.915086 2583 scope.go:117] "RemoveContainer" containerID="bb1b21a4bd78a584d0b11a80fe134b5ba9e2d2dbb8ca19d90e92692fbfdb2744" Sep 10 12:20:03.915562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:03.915504 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:20:03.915726 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:03.915706 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:20:09.961253 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:09.961218 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:20:09.961682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:09.961276 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:20:09.961682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:09.961297 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:20:09.961682 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:09.961311 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" Sep 10 12:20:09.961794 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:09.961774 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:20:09.961987 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:09.961966 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:20:14.896821 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:14.896787 2583 scope.go:117] "RemoveContainer" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" Sep 10 12:20:14.897218 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:14.896976 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:20:15.896669 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:15.896633 2583 scope.go:117] "RemoveContainer" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" Sep 10 12:20:15.896905 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:15.896866 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:20:22.897073 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:22.897039 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:20:22.897563 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:22.897211 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:20:25.897124 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:25.897085 2583 scope.go:117] "RemoveContainer" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" Sep 10 12:20:25.897528 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:25.897265 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:20:26.896637 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:26.896608 2583 scope.go:117] "RemoveContainer" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" Sep 10 12:20:26.896833 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:26.896813 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:20:34.896973 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:34.896940 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:20:34.897402 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:34.897134 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:20:38.897236 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:38.897200 2583 scope.go:117] "RemoveContainer" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" Sep 10 12:20:38.897646 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:38.897464 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:20:39.898509 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:39.898481 2583 scope.go:117] "RemoveContainer" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" Sep 10 12:20:39.898969 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:39.898662 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:20:47.896856 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:47.896818 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:20:47.897264 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:47.897033 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:20:51.896375 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:51.896314 2583 scope.go:117] "RemoveContainer" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" Sep 10 12:20:51.896773 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:51.896538 2583 scope.go:117] "RemoveContainer" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" Sep 10 12:20:51.896773 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:51.896744 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:20:52.161326 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:52.161301 2583 generic.go:356] "Generic (PLEG): container finished" podID="be8dde9d-0117-44f1-b835-2859431ab693" containerID="3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6" exitCode=0 Sep 10 12:20:52.161458 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:52.161385 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" event={"ID":"be8dde9d-0117-44f1-b835-2859431ab693","Type":"ContainerDied","Data":"3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6"} Sep 10 12:20:52.161458 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:52.161442 2583 scope.go:117] "RemoveContainer" containerID="21b3879d70269ad215a8e35b23c196a801580cd1394d91d9fe662d560908ee38" Sep 10 12:20:52.161869 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:52.161851 2583 scope.go:117] "RemoveContainer" containerID="3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6" Sep 10 12:20:52.162098 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:52.162076 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:20:53.849461 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:53.849424 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:20:53.849849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:53.849479 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:20:53.849849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:53.849493 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:20:53.849849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:53.849510 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" Sep 10 12:20:53.849849 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:20:53.849827 2583 scope.go:117] "RemoveContainer" containerID="3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6" Sep 10 12:20:53.850033 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:20:53.850014 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:21:01.896965 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:01.896923 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:21:01.897715 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:01.897683 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:21:04.897125 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:04.897091 2583 scope.go:117] "RemoveContainer" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" Sep 10 12:21:05.228034 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:05.228005 2583 generic.go:356] "Generic (PLEG): container finished" podID="3a339f96-7bfd-4023-9926-e4b60b5118f1" containerID="e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff" exitCode=0 Sep 10 12:21:05.228173 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:05.228080 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" event={"ID":"3a339f96-7bfd-4023-9926-e4b60b5118f1","Type":"ContainerDied","Data":"e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff"} Sep 10 12:21:05.228173 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:05.228134 2583 scope.go:117] "RemoveContainer" containerID="7e8f966a26b8a0ae785d87b1f5c46e383ffe7ac0327bb2d1a33911ada3108aab" Sep 10 12:21:05.228623 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:05.228606 2583 scope.go:117] "RemoveContainer" containerID="e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff" Sep 10 12:21:05.228876 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:05.228841 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:21:05.896810 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:05.896768 2583 scope.go:117] "RemoveContainer" containerID="3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6" Sep 10 12:21:05.897033 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:05.897006 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:21:09.443541 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:09.443504 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:21:09.444038 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:09.443584 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:21:09.444038 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:09.443606 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:21:09.444038 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:09.443614 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" Sep 10 12:21:09.444038 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:09.444013 2583 scope.go:117] "RemoveContainer" containerID="e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff" Sep 10 12:21:09.444249 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:09.444229 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:21:10.254426 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:10.254397 2583 scope.go:117] "RemoveContainer" containerID="e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff" Sep 10 12:21:10.254614 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:10.254571 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:21:13.896301 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:13.896215 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:21:13.896789 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:13.896437 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:21:18.896535 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:18.896500 2583 scope.go:117] "RemoveContainer" containerID="3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6" Sep 10 12:21:18.896953 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:18.896695 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:21:20.897139 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:20.897094 2583 scope.go:117] "RemoveContainer" containerID="e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff" Sep 10 12:21:20.897621 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:20.897345 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:21:26.896925 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:26.896894 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:21:26.897431 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:26.897092 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:21:32.397941 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:32.397909 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_be8dde9d-0117-44f1-b835-2859431ab693/main/5.log" Sep 10 12:21:32.519963 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:32.519929 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_be8dde9d-0117-44f1-b835-2859431ab693/main/5.log" Sep 10 12:21:33.897097 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:33.897067 2583 scope.go:117] "RemoveContainer" containerID="3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6" Sep 10 12:21:33.897508 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:33.897265 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:21:35.897138 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:35.897097 2583 scope.go:117] "RemoveContainer" containerID="e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff" Sep 10 12:21:35.897685 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:35.897279 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:21:39.899764 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:39.899733 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:21:39.900177 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:39.899915 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:21:40.211286 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:40.211207 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d9854688c-lxdsp_e1b992a3-2e13-4b3e-8781-bb86fdcc1a79/copy-manifests/0.log" Sep 10 12:21:40.222965 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:40.222920 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d9854688c-lxdsp_e1b992a3-2e13-4b3e-8781-bb86fdcc1a79/manager/0.log" Sep 10 12:21:40.849133 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:40.849103 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ts68j_91ad704e-0dd3-4ca3-9f56-6c77d50c5a86/postgres/0.log" Sep 10 12:21:42.190093 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:42.190059 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-62rlp_f6b306f6-7c17-4a37-864a-5d2fb22a759a/manager/0.log" Sep 10 12:21:43.169331 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:43.169301 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8784dff-k7lwp_b819d774-1d7e-4079-89be-cc8e15a64098/kube-auth-proxy/0.log" Sep 10 12:21:44.017599 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:44.017566 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-859856b874-x6ff7_b8f7a596-0d80-499a-b8ef-4a284d8fb973/main/5.log" Sep 10 12:21:44.391942 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:44.391913 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5ckbl_efc0d1d8-dc9b-4c65-93a6-e634a2cf43f1/main/0.log" Sep 10 12:21:44.511933 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:44.511901 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhtldc_185b3e98-9ae1-488c-9359-0142e755492b/main/0.log" Sep 10 12:21:44.637260 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:44.637232 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_be8dde9d-0117-44f1-b835-2859431ab693/main/5.log" Sep 10 12:21:44.757395 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:44.757283 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_3a339f96-7bfd-4023-9926-e4b60b5118f1/main/5.log" Sep 10 12:21:47.896542 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:47.896501 2583 scope.go:117] "RemoveContainer" containerID="3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6" Sep 10 12:21:47.897047 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:47.896560 2583 scope.go:117] "RemoveContainer" containerID="e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff" Sep 10 12:21:47.897047 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:47.896734 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:21:47.897047 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:47.896773 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:21:51.896573 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:51.896541 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:21:51.899002 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:51.896729 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:21:53.969254 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:53.969220 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7wxnx_faf4ab54-b2cc-42bf-a906-e23f8d7949d9/global-pull-secret-syncer/0.log" Sep 10 12:21:54.107091 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:54.107057 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lg9xf_227a6eda-79a6-49f9-9bed-eb32fbeff27a/konnectivity-agent/0.log" Sep 10 12:21:54.235414 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:54.235324 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-9.ec2.internal_4e79bb2805f50e77c363fc54fc65fe32/haproxy/0.log" Sep 10 12:21:57.805269 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:57.805237 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-62rlp_f6b306f6-7c17-4a37-864a-5d2fb22a759a/manager/0.log" Sep 10 12:21:59.624696 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.624658 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3/alertmanager/0.log" Sep 10 12:21:59.651875 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.651844 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3/config-reloader/0.log" Sep 10 12:21:59.679879 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.679850 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3/kube-rbac-proxy-web/0.log" Sep 10 12:21:59.708142 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.708110 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3/kube-rbac-proxy/0.log" Sep 10 12:21:59.734765 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.734732 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3/kube-rbac-proxy-metric/0.log" Sep 10 12:21:59.764334 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.764301 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3/prom-label-proxy/0.log" Sep 10 12:21:59.790073 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.790045 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a9503c8-ae0c-4a4e-bb89-35ff2f53faf3/init-config-reloader/0.log" Sep 10 12:21:59.858766 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.858679 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-t5j77_2d5d9d44-6619-4706-9689-b66c12635aa6/kube-state-metrics/0.log" Sep 10 12:21:59.883023 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.882991 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-t5j77_2d5d9d44-6619-4706-9689-b66c12635aa6/kube-rbac-proxy-main/0.log" Sep 10 12:21:59.898705 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.898673 2583 scope.go:117] "RemoveContainer" containerID="e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff" Sep 10 12:21:59.898904 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:21:59.898863 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:21:59.907445 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.907419 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-t5j77_2d5d9d44-6619-4706-9689-b66c12635aa6/kube-rbac-proxy-self/0.log" Sep 10 12:21:59.962025 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:21:59.961992 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-786964d67d-s7btn_ca347c44-8285-4f20-aaeb-663f8b27b0ee/metrics-server/0.log" Sep 10 12:22:00.001673 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.001643 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-xwr7b_b987202e-934b-4ad0-be06-489809b415ae/monitoring-plugin/0.log" Sep 10 12:22:00.230001 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.229970 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcjkz_3510d8b5-99e5-4726-a1f3-177d939fe640/node-exporter/0.log" Sep 10 12:22:00.257978 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.257949 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcjkz_3510d8b5-99e5-4726-a1f3-177d939fe640/kube-rbac-proxy/0.log" Sep 10 12:22:00.288562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.288529 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcjkz_3510d8b5-99e5-4726-a1f3-177d939fe640/init-textfile/0.log" Sep 10 12:22:00.321537 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.321502 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-fvs6v_152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab/kube-rbac-proxy-main/0.log" Sep 10 12:22:00.343654 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.343626 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-fvs6v_152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab/kube-rbac-proxy-self/0.log" Sep 10 12:22:00.371386 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.371345 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-fvs6v_152f4948-cfae-4ed7-8ca8-4cb3caa6c6ab/openshift-state-metrics/0.log" Sep 10 12:22:00.442029 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.441948 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22df79fa-6ca3-4868-b1f9-aaebd68032b3/prometheus/0.log" Sep 10 12:22:00.482276 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.482230 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22df79fa-6ca3-4868-b1f9-aaebd68032b3/config-reloader/0.log" Sep 10 12:22:00.515718 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.515692 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22df79fa-6ca3-4868-b1f9-aaebd68032b3/thanos-sidecar/0.log" Sep 10 12:22:00.543019 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.542989 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22df79fa-6ca3-4868-b1f9-aaebd68032b3/kube-rbac-proxy-web/0.log" Sep 10 12:22:00.578171 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.578145 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22df79fa-6ca3-4868-b1f9-aaebd68032b3/kube-rbac-proxy/0.log" Sep 10 12:22:00.627117 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.627076 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22df79fa-6ca3-4868-b1f9-aaebd68032b3/kube-rbac-proxy-thanos/0.log" Sep 10 12:22:00.654199 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.654168 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22df79fa-6ca3-4868-b1f9-aaebd68032b3/init-config-reloader/0.log" Sep 10 12:22:00.694934 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.694841 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-2lg8p_a8754300-e471-407b-bf99-0f7663363762/prometheus-operator/0.log" Sep 10 12:22:00.720074 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.720031 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-2lg8p_a8754300-e471-407b-bf99-0f7663363762/kube-rbac-proxy/0.log" Sep 10 12:22:00.810905 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.810874 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6bc9f66b54-4mllm_91fbbc13-01d5-46c7-95b9-d2f6f56b51af/telemeter-client/0.log" Sep 10 12:22:00.843498 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.843467 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6bc9f66b54-4mllm_91fbbc13-01d5-46c7-95b9-d2f6f56b51af/reload/0.log" Sep 10 12:22:00.866521 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.866486 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6bc9f66b54-4mllm_91fbbc13-01d5-46c7-95b9-d2f6f56b51af/kube-rbac-proxy/0.log" Sep 10 12:22:00.910496 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.910463 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d48ffc5b4-tbtpv_0512f0cb-b28b-4543-a8cc-7470058ebe9d/thanos-query/0.log" Sep 10 12:22:00.935124 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.935098 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d48ffc5b4-tbtpv_0512f0cb-b28b-4543-a8cc-7470058ebe9d/kube-rbac-proxy-web/0.log" Sep 10 12:22:00.961503 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.961420 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d48ffc5b4-tbtpv_0512f0cb-b28b-4543-a8cc-7470058ebe9d/kube-rbac-proxy/0.log" Sep 10 12:22:00.988485 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:00.988460 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d48ffc5b4-tbtpv_0512f0cb-b28b-4543-a8cc-7470058ebe9d/prom-label-proxy/0.log" Sep 10 12:22:01.033340 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:01.033307 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d48ffc5b4-tbtpv_0512f0cb-b28b-4543-a8cc-7470058ebe9d/kube-rbac-proxy-rules/0.log" Sep 10 12:22:01.069797 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:01.069769 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d48ffc5b4-tbtpv_0512f0cb-b28b-4543-a8cc-7470058ebe9d/kube-rbac-proxy-metrics/0.log" Sep 10 12:22:02.897070 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:02.897040 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:22:02.897559 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:02.897175 2583 scope.go:117] "RemoveContainer" containerID="3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6" Sep 10 12:22:02.897559 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:22:02.897224 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:22:02.897559 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:22:02.897429 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:22:03.930035 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:03.929997 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-6ftmg_cbbf08dd-74e3-476f-89f1-a883cb9b5054/download-server/0.log" Sep 10 12:22:04.757295 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.757264 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr"] Sep 10 12:22:04.761072 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.761044 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.763670 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.763648 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vgmhj\"/\"openshift-service-ca.crt\"" Sep 10 12:22:04.763816 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.763792 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vgmhj\"/\"default-dockercfg-sslq7\"" Sep 10 12:22:04.764258 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.764243 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vgmhj\"/\"kube-root-ca.crt\"" Sep 10 12:22:04.772886 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.772863 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr"] Sep 10 12:22:04.820374 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.820311 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsprb\" (UniqueName: \"kubernetes.io/projected/f88a6c02-c347-40a3-a430-1c3b6d269e39-kube-api-access-tsprb\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.820562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.820383 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-sys\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.820562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.820415 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-proc\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.820562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.820490 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-podres\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.820562 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.820516 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-lib-modules\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.921073 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.921026 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-lib-modules\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.921235 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.921120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsprb\" (UniqueName: \"kubernetes.io/projected/f88a6c02-c347-40a3-a430-1c3b6d269e39-kube-api-access-tsprb\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.921235 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.921152 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-sys\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.921235 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.921176 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-proc\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.921463 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.921238 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-lib-modules\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.921463 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.921255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-podres\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.921463 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.921265 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-sys\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.921463 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.921392 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-proc\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.921463 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.921400 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f88a6c02-c347-40a3-a430-1c3b6d269e39-podres\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:04.931379 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:04.931338 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsprb\" (UniqueName: \"kubernetes.io/projected/f88a6c02-c347-40a3-a430-1c3b6d269e39-kube-api-access-tsprb\") pod \"perf-node-gather-daemonset-2vrhr\" (UID: \"f88a6c02-c347-40a3-a430-1c3b6d269e39\") " pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:05.072044 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:05.072014 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:05.353081 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:05.353023 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr"] Sep 10 12:22:05.355715 ip-10-0-141-9 kubenswrapper[2583]: W0910 12:22:05.355685 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf88a6c02_c347_40a3_a430_1c3b6d269e39.slice/crio-64564b6148b5766c53b93520a7fdf23f23b812591608ccbdd6c6502543402616 WatchSource:0}: Error finding container 64564b6148b5766c53b93520a7fdf23f23b812591608ccbdd6c6502543402616: Status 404 returned error can't find the container with id 64564b6148b5766c53b93520a7fdf23f23b812591608ccbdd6c6502543402616 Sep 10 12:22:05.527844 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:05.527805 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" event={"ID":"f88a6c02-c347-40a3-a430-1c3b6d269e39","Type":"ContainerStarted","Data":"43d35f184849767522c2d376fde7a6f42264cc774e47dd27c25aea9dddb72df9"} Sep 10 12:22:05.528021 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:05.527857 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" event={"ID":"f88a6c02-c347-40a3-a430-1c3b6d269e39","Type":"ContainerStarted","Data":"64564b6148b5766c53b93520a7fdf23f23b812591608ccbdd6c6502543402616"} Sep 10 12:22:05.528021 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:05.527888 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:05.898372 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:05.898333 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-55t8l_f15ce4a4-4f2e-4b78-9c48-3c169c83439a/dns/0.log" Sep 10 12:22:05.947933 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:05.947900 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-55t8l_f15ce4a4-4f2e-4b78-9c48-3c169c83439a/kube-rbac-proxy/0.log" Sep 10 12:22:06.157699 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:06.157614 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-klmnw_dcea16cd-6eca-4e99-8c4e-f5d1f7d956e3/dns-node-resolver/0.log" Sep 10 12:22:06.699338 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:06.699307 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bxjqj_1dfb8b69-72b9-4fb8-b33d-a2b938b653a1/node-ca/0.log" Sep 10 12:22:07.824713 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:07.824683 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8784dff-k7lwp_b819d774-1d7e-4079-89be-cc8e15a64098/kube-auth-proxy/0.log" Sep 10 12:22:08.587415 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:08.587382 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nsznl_a368f756-d49b-4873-8ae0-108960413ac6/serve-healthcheck-canary/0.log" Sep 10 12:22:09.339987 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:09.339947 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bkljw_52d6c263-5574-4700-81b6-e4d979cc71c3/kube-rbac-proxy/0.log" Sep 10 12:22:09.362293 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:09.362267 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bkljw_52d6c263-5574-4700-81b6-e4d979cc71c3/exporter/0.log" Sep 10 12:22:09.390618 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:09.390578 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bkljw_52d6c263-5574-4700-81b6-e4d979cc71c3/extractor/0.log" Sep 10 12:22:11.544450 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:11.544413 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" Sep 10 12:22:11.566442 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:11.566388 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vgmhj/perf-node-gather-daemonset-2vrhr" podStartSLOduration=7.56636968 podStartE2EDuration="7.56636968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:22:05.549254555 +0000 UTC m=+946.126578163" watchObservedRunningTime="2026-09-10 12:22:11.56636968 +0000 UTC m=+952.143693282" Sep 10 12:22:11.883063 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:11.882962 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d9854688c-lxdsp_e1b992a3-2e13-4b3e-8781-bb86fdcc1a79/manager/0.log" Sep 10 12:22:11.907465 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:11.907439 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d9854688c-lxdsp_e1b992a3-2e13-4b3e-8781-bb86fdcc1a79/copy-manifests/0.log" Sep 10 12:22:13.258547 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:13.258513 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b86f6cfc4-2k4dw_e3e6d498-0777-4a32-9821-d043d1575448/manager/0.log" Sep 10 12:22:13.600842 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:13.600740 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8784dff-k7lwp_b819d774-1d7e-4079-89be-cc8e15a64098/kube-auth-proxy/0.log" Sep 10 12:22:13.896487 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:13.896450 2583 scope.go:117] "RemoveContainer" containerID="3f0f828da08be700982e564068da28c404463e44eabc31073a47ad116c1c75c6" Sep 10 12:22:13.896666 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:13.896512 2583 scope.go:117] "RemoveContainer" containerID="e41cfa261b0ade83d09fd56847ad7c52813c7ab0cb3f958eb282412d9585eaff" Sep 10 12:22:13.896726 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:22:13.896676 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-t79nw_llm(3a339f96-7bfd-4023-9926-e4b60b5118f1)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-t79nw" podUID="3a339f96-7bfd-4023-9926-e4b60b5118f1" Sep 10 12:22:13.896726 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:22:13.896689 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq_llm(be8dde9d-0117-44f1-b835-2859431ab693)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l9htq" podUID="be8dde9d-0117-44f1-b835-2859431ab693" Sep 10 12:22:14.897053 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:14.897018 2583 scope.go:117] "RemoveContainer" containerID="b763c761823381b1d8cebbc4370fd36b0a23fa65bde2d6534cbab2e3714a5123" Sep 10 12:22:14.897486 ip-10-0-141-9 kubenswrapper[2583]: E0910 12:22:14.897288 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-x6ff7_llm(b8f7a596-0d80-499a-b8ef-4a284d8fb973)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-x6ff7" podUID="b8f7a596-0d80-499a-b8ef-4a284d8fb973" Sep 10 12:22:15.057705 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:15.057670 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8784dff-k7lwp_b819d774-1d7e-4079-89be-cc8e15a64098/kube-auth-proxy/0.log" Sep 10 12:22:20.840661 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:20.840609 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsdv2_31074e24-3342-4c93-94a8-9abc7e0e454e/kube-multus-additional-cni-plugins/0.log" Sep 10 12:22:20.864519 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:20.864490 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsdv2_31074e24-3342-4c93-94a8-9abc7e0e454e/egress-router-binary-copy/0.log" Sep 10 12:22:20.882922 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:20.882896 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsdv2_31074e24-3342-4c93-94a8-9abc7e0e454e/cni-plugins/0.log" Sep 10 12:22:20.901575 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:20.901548 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsdv2_31074e24-3342-4c93-94a8-9abc7e0e454e/bond-cni-plugin/0.log" Sep 10 12:22:20.920262 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:20.920233 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsdv2_31074e24-3342-4c93-94a8-9abc7e0e454e/routeoverride-cni/0.log" Sep 10 12:22:20.940104 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:20.940078 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsdv2_31074e24-3342-4c93-94a8-9abc7e0e454e/whereabouts-cni-bincopy/0.log" Sep 10 12:22:20.959434 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:20.959282 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsdv2_31074e24-3342-4c93-94a8-9abc7e0e454e/whereabouts-cni/0.log" Sep 10 12:22:21.165823 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:21.165791 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fb9x6_0eb91a8b-4cf7-4e08-b9e3-f5d46b6b5108/kube-multus/0.log" Sep 10 12:22:21.307261 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:21.307226 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s8hwl_35e290b1-c39f-4ab8-b6ed-9584e49b3ae4/network-metrics-daemon/0.log" Sep 10 12:22:21.324396 ip-10-0-141-9 kubenswrapper[2583]: I0910 12:22:21.324341 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s8hwl_35e290b1-c39f-4ab8-b6ed-9584e49b3ae4/kube-rbac-proxy/0.log"