Sep 15 07:25:08.069755 ip-10-0-135-38 systemd[1]: Starting Kubernetes Kubelet... Sep 15 07:25:08.432470 ip-10-0-135-38 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 07:25:08.432470 ip-10-0-135-38 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 15 07:25:08.432470 ip-10-0-135-38 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 07:25:08.432470 ip-10-0-135-38 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 15 07:25:08.432470 ip-10-0-135-38 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 07:25:08.434202 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.434097 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 15 07:25:08.439036 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439021 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:25:08.439036 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439036 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439040 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439044 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439047 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439050 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439053 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439058 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439063 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439067 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439070 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439073 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439076 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439079 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439082 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439084 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439087 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439090 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439092 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439095 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:25:08.439097 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439098 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439101 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439103 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439107 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439110 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439113 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439115 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439118 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439121 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439123 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439126 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439129 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439131 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439134 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439137 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439139 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439142 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439144 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439147 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439150 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439152 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:25:08.439560 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439154 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439157 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439160 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439162 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439165 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439168 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439171 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439173 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439176 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439179 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439181 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439184 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439187 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439190 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439193 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439195 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439198 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439201 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439204 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439206 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:25:08.440084 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439209 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439211 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439214 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439216 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439219 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439222 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439224 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439227 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439229 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439232 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439241 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439244 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439247 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439250 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439254 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439258 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439261 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439264 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439267 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:25:08.440587 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439269 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439272 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439275 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439277 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439280 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439283 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439287 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439290 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439293 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439295 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439298 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439301 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439303 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439306 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439309 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439311 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439314 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439317 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.439320 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:25:08.441864 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440089 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440096 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440100 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440103 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440107 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440111 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440114 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440117 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440120 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440123 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440127 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440129 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440134 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440137 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440140 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440143 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440145 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440156 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440159 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440162 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:25:08.442355 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440164 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440167 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440170 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440173 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440175 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440178 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440180 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440183 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440186 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440188 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440190 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440193 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440196 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440199 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440201 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440205 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440209 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440212 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440216 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:25:08.442872 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440219 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440221 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440224 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440226 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440229 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440231 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440234 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440237 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440240 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440242 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440245 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440247 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440250 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440252 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440255 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440257 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440262 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440265 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440268 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440271 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:25:08.443346 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440274 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440276 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440279 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440284 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440288 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440291 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440294 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440296 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440300 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440303 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440306 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440309 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440313 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440315 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440318 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440321 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440323 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440326 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440329 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:25:08.443862 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440331 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440334 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440336 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440339 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440341 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440343 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440346 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440348 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440351 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440354 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440357 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440359 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440362 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440364 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440367 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440369 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440372 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440375 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440377 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440379 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:25:08.444338 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440382 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440779 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440786 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440789 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440792 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440796 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440799 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440802 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440805 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440808 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440811 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440814 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440817 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440820 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440822 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440825 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440827 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440830 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440833 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440835 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:25:08.444838 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440838 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440842 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440846 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440849 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440852 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440854 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440857 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440859 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440862 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440865 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440867 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440870 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440873 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440876 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440879 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440881 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440884 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440886 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440889 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:25:08.445314 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440892 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440896 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440900 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440902 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440905 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440909 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440911 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440914 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440916 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440918 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440921 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440923 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440926 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440928 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440931 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440934 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440937 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440939 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440941 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:25:08.445793 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440944 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440946 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440949 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440952 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440955 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440957 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440963 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440966 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440968 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440971 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440973 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440976 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440978 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440981 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440983 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440988 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440992 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440995 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.440998 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441000 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:25:08.446255 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441003 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441005 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441008 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441011 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441013 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441016 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441018 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441021 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441024 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441026 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441029 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441031 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441034 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441036 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441039 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441041 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441044 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441046 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441050 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441053 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:25:08.446755 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441055 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441058 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441128 2577 flags.go:64] FLAG: --address="0.0.0.0" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441161 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441170 2577 flags.go:64] FLAG: --anonymous-auth="true" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441178 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441183 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441186 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441190 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441194 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441197 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441200 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441203 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441207 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441210 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441213 2577 flags.go:64] FLAG: --cgroup-root="" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441216 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441219 2577 flags.go:64] FLAG: --client-ca-file="" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441221 2577 flags.go:64] FLAG: --cloud-provider="external" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441226 2577 flags.go:64] FLAG: --cluster-dns="[]" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441231 2577 flags.go:64] FLAG: --cluster-domain="" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441233 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441236 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441240 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 15 07:25:08.447276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441243 2577 flags.go:64] FLAG: --container-log-max-files="5" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441247 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441250 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441253 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441256 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441259 2577 flags.go:64] FLAG: --contention-profiling="false" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441263 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441266 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441270 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441272 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441277 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441279 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441282 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441285 2577 flags.go:64] FLAG: --enable-load-reader="false" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441288 2577 flags.go:64] FLAG: --enable-server="true" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441291 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441296 2577 flags.go:64] FLAG: --event-burst="100" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441299 2577 flags.go:64] FLAG: --event-qps="50" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441302 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441305 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441307 2577 flags.go:64] FLAG: --eviction-hard="" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441311 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441314 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441317 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441320 2577 flags.go:64] FLAG: --eviction-soft="" Sep 15 07:25:08.447869 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441322 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441325 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441330 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441333 2577 flags.go:64] FLAG: --experimental-mounter-path="" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441336 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441339 2577 flags.go:64] FLAG: --fail-swap-on="true" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441342 2577 flags.go:64] FLAG: --feature-gates="" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441345 2577 flags.go:64] FLAG: --file-check-frequency="20s" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441348 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441351 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441354 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441357 2577 flags.go:64] FLAG: --healthz-port="10248" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441360 2577 flags.go:64] FLAG: --help="false" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441364 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441367 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441370 2577 flags.go:64] FLAG: --http-check-frequency="20s" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441373 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441377 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441380 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441383 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441385 2577 flags.go:64] FLAG: --image-service-endpoint="" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441388 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441391 2577 flags.go:64] FLAG: --kube-api-burst="100" Sep 15 07:25:08.448468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441402 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441406 2577 flags.go:64] FLAG: --kube-api-qps="50" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441408 2577 flags.go:64] FLAG: --kube-reserved="" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441411 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441414 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441417 2577 flags.go:64] FLAG: --kubelet-cgroups="" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441420 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441423 2577 flags.go:64] FLAG: --lock-file="" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441426 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441429 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441432 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441442 2577 flags.go:64] FLAG: --log-json-split-stream="false" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441445 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441448 2577 flags.go:64] FLAG: --log-text-split-stream="false" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441450 2577 flags.go:64] FLAG: --logging-format="text" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441453 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441457 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441460 2577 flags.go:64] FLAG: --manifest-url="" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441463 2577 flags.go:64] FLAG: --manifest-url-header="" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441467 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441470 2577 flags.go:64] FLAG: --max-open-files="1000000" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441474 2577 flags.go:64] FLAG: --max-pods="110" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441493 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441496 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441499 2577 flags.go:64] FLAG: --memory-manager-policy="None" Sep 15 07:25:08.449081 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441502 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441506 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441509 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441511 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441519 2577 flags.go:64] FLAG: --node-status-max-images="50" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441522 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441525 2577 flags.go:64] FLAG: --oom-score-adj="-999" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441528 2577 flags.go:64] FLAG: --pod-cidr="" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441538 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441543 2577 flags.go:64] FLAG: --pod-manifest-path="" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441546 2577 flags.go:64] FLAG: --pod-max-pids="-1" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441549 2577 flags.go:64] FLAG: --pods-per-core="0" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441552 2577 flags.go:64] FLAG: --port="10250" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441556 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441558 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02bb9088bc069af7e" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441562 2577 flags.go:64] FLAG: --qos-reserved="" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441565 2577 flags.go:64] FLAG: --read-only-port="10255" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441568 2577 flags.go:64] FLAG: --register-node="true" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441572 2577 flags.go:64] FLAG: --register-with-taints="" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441576 2577 flags.go:64] FLAG: --registry-burst="10" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441578 2577 flags.go:64] FLAG: --registry-qps="5" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441581 2577 flags.go:64] FLAG: --reserved-cpus="" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441584 2577 flags.go:64] FLAG: --reserved-memory="" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441588 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441591 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 15 07:25:08.449722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441594 2577 flags.go:64] FLAG: --rotate-certificates="false" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441597 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441600 2577 flags.go:64] FLAG: --runonce="false" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441603 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441606 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441611 2577 flags.go:64] FLAG: --seccomp-default="false" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441614 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441617 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441620 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441623 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441626 2577 flags.go:64] FLAG: --storage-driver-password="root" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441628 2577 flags.go:64] FLAG: --storage-driver-secure="false" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441631 2577 flags.go:64] FLAG: --storage-driver-table="stats" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441634 2577 flags.go:64] FLAG: --storage-driver-user="root" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441637 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441639 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441648 2577 flags.go:64] FLAG: --system-cgroups="" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441651 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441657 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441659 2577 flags.go:64] FLAG: --tls-cert-file="" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441662 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441666 2577 flags.go:64] FLAG: --tls-min-version="" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441669 2577 flags.go:64] FLAG: --tls-private-key-file="" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441671 2577 flags.go:64] FLAG: --topology-manager-policy="none" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441674 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 15 07:25:08.450318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441678 2577 flags.go:64] FLAG: --topology-manager-scope="container" Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441681 2577 flags.go:64] FLAG: --v="2" Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441685 2577 flags.go:64] FLAG: --version="false" Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441689 2577 flags.go:64] FLAG: --vmodule="" Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441692 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.441695 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441806 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441810 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441813 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441815 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441818 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441820 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441825 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441827 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441830 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441833 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441835 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441838 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441841 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441844 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441847 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:25:08.450983 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441849 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441853 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441857 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441861 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441865 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441868 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441871 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441874 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441877 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441879 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441882 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441886 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441889 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441891 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441894 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441896 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441899 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441901 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:25:08.451661 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441905 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441909 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441911 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441914 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441917 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441921 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441924 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441926 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441929 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441932 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441934 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441937 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441940 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441943 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441945 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441948 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441950 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441953 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441956 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441959 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:25:08.452106 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441961 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441964 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441966 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441969 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441971 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441975 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441978 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441980 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441983 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441986 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441988 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441991 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441993 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441996 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.441998 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442001 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442004 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442007 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442010 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442012 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:25:08.452618 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442015 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442017 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442020 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442022 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442025 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442028 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442030 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442033 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442036 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442038 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442041 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442044 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442046 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442049 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442052 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442054 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442057 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442061 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442064 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442067 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:25:08.453118 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442069 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442072 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442075 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442077 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442080 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.442082 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.442722 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.450025 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.450144 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.450866 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.450878 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450936 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450940 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450944 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450947 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:25:08.453621 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450950 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450952 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450955 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450958 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450960 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450963 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450966 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450969 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450981 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450983 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450986 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450989 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450992 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.450995 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451000 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451004 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451007 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451010 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451013 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:25:08.453995 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451016 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451018 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451021 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451024 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451027 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451030 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451033 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451036 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451039 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451041 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451044 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451047 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451049 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451052 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451054 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451057 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451060 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451062 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451065 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:25:08.454474 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451067 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451070 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451073 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451076 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451080 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451082 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451085 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451088 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451090 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451093 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451095 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451098 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451101 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451103 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451106 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451109 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451112 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451114 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451117 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451120 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:25:08.454944 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451122 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451125 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451127 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451130 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451133 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451135 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451138 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451140 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451143 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451147 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451150 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451153 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451155 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451158 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451161 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451163 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451168 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451170 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451173 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:25:08.455432 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451176 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451179 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451183 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451185 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451188 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451191 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451194 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451196 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451199 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451202 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451204 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451207 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451210 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451212 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451215 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451218 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451220 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:25:08.455965 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451223 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.451228 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451337 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451342 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451346 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451349 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451351 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451354 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451357 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451359 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451362 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451364 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451367 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451370 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451373 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:25:08.456390 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451375 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451378 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451381 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451383 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451386 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451388 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451391 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451394 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451396 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451399 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451402 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451404 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451407 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451409 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451412 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451415 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451417 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451420 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451423 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451426 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:25:08.456765 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451428 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451430 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451433 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451435 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451438 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451440 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451443 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451445 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451448 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451452 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451455 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451458 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451461 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451464 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451467 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451470 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451473 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451475 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451491 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:25:08.457245 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451494 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451497 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451499 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451502 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451505 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451508 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451510 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451513 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451516 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451519 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451521 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451524 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451527 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451531 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451534 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451537 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451540 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451543 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451545 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451548 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:25:08.457716 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451551 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451553 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451556 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451558 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451561 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451564 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451567 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451569 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451572 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451574 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451579 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451582 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451585 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451588 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451590 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451593 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451595 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451598 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451601 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451603 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:25:08.458309 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451606 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451608 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451611 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451614 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451616 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451619 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:08.451621 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.451626 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.451716 2577 server.go:962] "Client rotation is on, will bootstrap in background" Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.454001 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.455188 2577 server.go:1019] "Starting client certificate rotation" Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.455292 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.455329 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 07:25:08.458813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.457471 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 07:25:08.462657 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.462633 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 07:25:08.468456 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.468439 2577 log.go:25] "Validated CRI v1 runtime API" Sep 15 07:25:08.472697 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.472683 2577 log.go:25] "Validated CRI v1 image API" Sep 15 07:25:08.473523 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.473506 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 15 07:25:08.477361 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.477333 2577 fs.go:135] Filesystem UUIDs: map[03d404bc-3d42-46e9-9b2e-06903281327c:/dev/nvme0n1p3 242caaf3-b683-49ad-93c9-24a5b8027eaa:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 15 07:25:08.477443 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.477360 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 15 07:25:08.481619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.481511 2577 manager.go:216] Machine: {Timestamp:2026-09-15 07:25:08.480982996 +0000 UTC m=+0.303454392 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3110748 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2358bc813890f0bf1bd46cb9ea7793 SystemUUID:ec2358bc-8138-90f0-bf1b-d46cb9ea7793 BootID:a1545a47-ad0b-4dc5-97d3-bff6f81eb311 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c8:d4:83:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c8:d4:83:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:17:1b:93:bd:fd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 15 07:25:08.481619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.481609 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 15 07:25:08.481788 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.481732 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 15 07:25:08.484365 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.484334 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 15 07:25:08.484536 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.484368 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-38.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 15 07:25:08.485200 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.485188 2577 topology_manager.go:138] "Creating topology manager with none policy" Sep 15 07:25:08.485252 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.485205 2577 container_manager_linux.go:309] "Creating device plugin manager" Sep 15 07:25:08.485252 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.485215 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 07:25:08.485829 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.485812 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 07:25:08.485953 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.485941 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 07:25:08.487011 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.486999 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 15 07:25:08.490156 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.490142 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 15 07:25:08.490290 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.490280 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 15 07:25:08.490353 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.490327 2577 kubelet.go:486] "Attempting to sync node with API server" Sep 15 07:25:08.490353 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.490351 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 15 07:25:08.490445 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.490375 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 15 07:25:08.490445 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.490389 2577 kubelet.go:398] "Adding apiserver pod source" Sep 15 07:25:08.490445 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.490402 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 15 07:25:08.490838 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.490823 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 07:25:08.490902 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.490853 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 07:25:08.496441 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.493687 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 15 07:25:08.497217 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.497200 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 15 07:25:08.497277 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.497237 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 15 07:25:08.499220 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499206 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499224 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499230 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499235 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499243 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499249 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499262 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499267 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499274 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499279 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499288 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 15 07:25:08.499289 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499293 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 15 07:25:08.499715 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499316 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 15 07:25:08.499715 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.499322 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 15 07:25:08.502334 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.502318 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-38.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 15 07:25:08.502417 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.502384 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-38.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 15 07:25:08.502582 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.502564 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 15 07:25:08.503189 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.503177 2577 server.go:1268] "Started kubelet" Sep 15 07:25:08.503292 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.503265 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 15 07:25:08.503348 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.503270 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 15 07:25:08.503348 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.503329 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 15 07:25:08.504052 ip-10-0-135-38 systemd[1]: Started Kubernetes Kubelet. Sep 15 07:25:08.504875 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.504858 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 15 07:25:08.504982 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.504968 2577 server.go:310] "Adding debug handlers to kubelet server" Sep 15 07:25:08.508520 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.508468 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 15 07:25:08.508883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.508869 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qv6bd" Sep 15 07:25:08.510233 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.509330 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-38.ec2.internal.18d56dc25d20bebe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-38.ec2.internal,UID:ip-10-0-135-38.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-38.ec2.internal,},FirstTimestamp:2026-09-15 07:25:08.50315027 +0000 UTC m=+0.325621666,LastTimestamp:2026-09-15 07:25:08.50315027 +0000 UTC m=+0.325621666,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-38.ec2.internal,}" Sep 15 07:25:08.513263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.513246 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 15 07:25:08.513905 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.513888 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 15 07:25:08.514083 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.514062 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:08.514428 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.514397 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 15 07:25:08.514534 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.514403 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 15 07:25:08.514534 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.514444 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 15 07:25:08.514642 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.514580 2577 reconstruct.go:97] "Volume reconstruction finished" Sep 15 07:25:08.514642 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.514589 2577 reconciler.go:29] "Reconciler: start to sync state" Sep 15 07:25:08.514800 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.514785 2577 factory.go:55] Registering systemd factory Sep 15 07:25:08.514854 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.514804 2577 factory.go:223] Registration of the systemd container factory successfully Sep 15 07:25:08.514854 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.514823 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qv6bd" Sep 15 07:25:08.515001 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.514988 2577 factory.go:153] Registering CRI-O factory Sep 15 07:25:08.515001 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.515002 2577 factory.go:223] Registration of the crio container factory successfully Sep 15 07:25:08.515093 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.515040 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 15 07:25:08.515093 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.515053 2577 factory.go:103] Registering Raw factory Sep 15 07:25:08.515093 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.515064 2577 manager.go:1196] Started watching for new ooms in manager Sep 15 07:25:08.515438 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.515424 2577 manager.go:349] Starting recovery of all containers Sep 15 07:25:08.517300 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.517277 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-38.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 15 07:25:08.517402 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.517367 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 15 07:25:08.526098 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.526077 2577 manager.go:354] Recovery completed Sep 15 07:25:08.531409 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.531396 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:25:08.533191 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.533175 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:25:08.533251 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.533202 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:25:08.533251 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.533212 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:25:08.533784 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.533771 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 15 07:25:08.533784 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.533782 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 15 07:25:08.533872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.533796 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 15 07:25:08.533872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.533833 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:25:08.534473 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.534457 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:25:08.534473 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.534505 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:25:08.534649 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.534521 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:25:08.536007 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.535994 2577 policy_none.go:49] "None policy: Start" Sep 15 07:25:08.536007 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.536009 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 15 07:25:08.536095 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.536020 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 15 07:25:08.537062 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.537047 2577 policy_none.go:47] "Start" Sep 15 07:25:08.563150 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.563133 2577 manager.go:337] "Starting Device Plugin manager" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.563174 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.563186 2577 server.go:87] "Starting device plugin registration server" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.563451 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.563463 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.563734 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.563805 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.563814 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.564253 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.564300 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.663938 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.667282 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.667339 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.667350 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.667374 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.675016 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.680619 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.675035 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-38.ec2.internal\": node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:08.682747 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.682688 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 15 07:25:08.684074 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.684056 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 15 07:25:08.684140 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.684113 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 15 07:25:08.684140 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.684136 2577 kubelet.go:2448] "Starting kubelet main sync loop" Sep 15 07:25:08.684241 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.684181 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 15 07:25:08.686189 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.686172 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:25:08.688858 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.688836 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:08.785174 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.785140 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal"] Sep 15 07:25:08.785267 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.785231 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:25:08.786128 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.786114 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:25:08.786192 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.786142 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:25:08.786192 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.786155 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:25:08.787718 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.787705 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:25:08.787840 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.787825 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.787877 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.787867 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:25:08.788414 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.788401 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:25:08.788492 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.788425 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:25:08.788492 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.788434 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:25:08.788570 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.788401 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:25:08.788570 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.788520 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:25:08.788570 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.788530 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:25:08.789304 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.789289 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:08.789769 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.789754 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.789846 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.789779 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:25:08.790407 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.790391 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:25:08.790478 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.790417 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:25:08.790478 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.790426 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:25:08.811141 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.811118 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-38.ec2.internal\" not found" node="ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.815556 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.815535 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-38.ec2.internal\" not found" node="ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.816568 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.816552 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b9dc28918bbb8a1972a191b9a118ed97-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal\" (UID: \"b9dc28918bbb8a1972a191b9a118ed97\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.816638 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.816582 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9dc28918bbb8a1972a191b9a118ed97-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal\" (UID: \"b9dc28918bbb8a1972a191b9a118ed97\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.816638 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.816611 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1bc2623acdeb0a77dc30a9def2462fc4-config\") pod \"kube-apiserver-proxy-ip-10-0-135-38.ec2.internal\" (UID: \"1bc2623acdeb0a77dc30a9def2462fc4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.889494 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.889463 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:08.917659 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.917637 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b9dc28918bbb8a1972a191b9a118ed97-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal\" (UID: \"b9dc28918bbb8a1972a191b9a118ed97\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.917762 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.917664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9dc28918bbb8a1972a191b9a118ed97-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal\" (UID: \"b9dc28918bbb8a1972a191b9a118ed97\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.917762 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.917688 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1bc2623acdeb0a77dc30a9def2462fc4-config\") pod \"kube-apiserver-proxy-ip-10-0-135-38.ec2.internal\" (UID: \"1bc2623acdeb0a77dc30a9def2462fc4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.917762 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.917723 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9dc28918bbb8a1972a191b9a118ed97-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal\" (UID: \"b9dc28918bbb8a1972a191b9a118ed97\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.917762 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.917734 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b9dc28918bbb8a1972a191b9a118ed97-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal\" (UID: \"b9dc28918bbb8a1972a191b9a118ed97\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.917910 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:08.917790 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1bc2623acdeb0a77dc30a9def2462fc4-config\") pod \"kube-apiserver-proxy-ip-10-0-135-38.ec2.internal\" (UID: \"1bc2623acdeb0a77dc30a9def2462fc4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal" Sep 15 07:25:08.989943 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:08.989878 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:09.090533 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:09.090495 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:09.114688 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.114665 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" Sep 15 07:25:09.119993 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.119971 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal" Sep 15 07:25:09.191472 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:09.191429 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:09.291945 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:09.291881 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:09.336512 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.336492 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:25:09.392829 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:09.392783 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:09.457121 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.457099 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 15 07:25:09.457588 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.457260 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 15 07:25:09.457588 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.457273 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 15 07:25:09.493196 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:09.493159 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-38.ec2.internal\" not found" Sep 15 07:25:09.515074 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.515055 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:25:09.519776 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.519750 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 07:20:09 +0000 UTC" deadline="2028-06-21 10:03:07.26775989 +0000 UTC" Sep 15 07:25:09.519776 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.519775 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15482h37m57.747988256s" Sep 15 07:25:09.532777 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.532747 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:25:09.614619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.614561 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" Sep 15 07:25:09.631935 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.631913 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 07:25:09.632891 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.632871 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal" Sep 15 07:25:09.642141 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.642122 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 07:25:09.761165 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:09.761002 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9dc28918bbb8a1972a191b9a118ed97.slice/crio-09861d3a1933139d726a82b511347452733c1d423d556362dd72d809ca6af47d WatchSource:0}: Error finding container 09861d3a1933139d726a82b511347452733c1d423d556362dd72d809ca6af47d: Status 404 returned error can't find the container with id 09861d3a1933139d726a82b511347452733c1d423d556362dd72d809ca6af47d Sep 15 07:25:09.761566 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:09.761538 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc2623acdeb0a77dc30a9def2462fc4.slice/crio-0a64710a1539198bcf3ff32c433549cb8c8904c4f9c97a62bdcbc0fbb0b62178 WatchSource:0}: Error finding container 0a64710a1539198bcf3ff32c433549cb8c8904c4f9c97a62bdcbc0fbb0b62178: Status 404 returned error can't find the container with id 0a64710a1539198bcf3ff32c433549cb8c8904c4f9c97a62bdcbc0fbb0b62178 Sep 15 07:25:09.766930 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:09.766915 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 07:25:10.493145 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.493114 2577 apiserver.go:52] "Watching apiserver" Sep 15 07:25:10.503173 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.503150 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 15 07:25:10.503594 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.503571 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-22spd","openshift-multus/network-metrics-daemon-f9mmh","openshift-network-diagnostics/network-check-target-m5skf","openshift-network-operator/iptables-alerter-qmdkl","openshift-ovn-kubernetes/ovnkube-node-4k5xc","kube-system/global-pull-secret-syncer-v8tt9","kube-system/konnectivity-agent-l6k5h","kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst","openshift-cluster-node-tuning-operator/tuned-8zr7v","openshift-dns/node-resolver-96dhs","openshift-image-registry/node-ca-k8pc6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal","openshift-multus/multus-2vds4"] Sep 15 07:25:10.506262 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.506247 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:10.507393 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.507372 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.508507 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.508487 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:10.508624 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.508550 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:10.509659 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.509642 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.509758 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.509728 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 15 07:25:10.510063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.510024 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ls7zq\"" Sep 15 07:25:10.511387 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.511368 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.511551 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.511533 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 15 07:25:10.511644 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.511583 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zkkwx\"" Sep 15 07:25:10.511644 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.511604 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 15 07:25:10.512078 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.512057 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 15 07:25:10.512830 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.512810 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 15 07:25:10.512931 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.512874 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v24rt\"" Sep 15 07:25:10.513186 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.513166 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:10.513274 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.513186 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 15 07:25:10.513274 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.513244 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:10.513508 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.513473 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 15 07:25:10.514403 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.513872 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 15 07:25:10.514403 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.514093 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 15 07:25:10.514403 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.514229 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.515147 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.515127 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 15 07:25:10.515312 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.515128 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-85b4z\"" Sep 15 07:25:10.515384 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.515314 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 15 07:25:10.516349 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.516317 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.517419 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.517392 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 15 07:25:10.517848 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.517826 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 15 07:25:10.517990 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.517969 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zvsds\"" Sep 15 07:25:10.518326 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.518309 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 15 07:25:10.518837 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.518615 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 15 07:25:10.519045 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.519017 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 15 07:25:10.519158 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.519131 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 15 07:25:10.520698 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.519432 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 15 07:25:10.520698 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.519669 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:10.520698 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.519770 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:10.520698 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.519916 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 15 07:25:10.520698 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.520631 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 15 07:25:10.520698 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.520640 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bn44g\"" Sep 15 07:25:10.521201 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.521120 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 15 07:25:10.523779 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.523745 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.525272 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.525254 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.526585 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.526569 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.526931 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.526909 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 15 07:25:10.527272 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.527244 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 15 07:25:10.527413 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.527389 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-79mhx\"" Sep 15 07:25:10.527940 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.527920 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 15 07:25:10.528444 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528425 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2x725\"" Sep 15 07:25:10.528566 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528448 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w74lm\" (UniqueName: \"kubernetes.io/projected/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-kube-api-access-w74lm\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.528566 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528480 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-var-lib-openvswitch\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.528566 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528508 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-conf-dir\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.528566 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-registration-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.528782 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20607e49-0155-468e-b8d4-0fda61b3be41-iptables-alerter-script\") pod \"iptables-alerter-qmdkl\" (UID: \"20607e49-0155-468e-b8d4-0fda61b3be41\") " pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.528782 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528606 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 15 07:25:10.528782 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528652 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-run-netns\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.528782 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528711 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-systemd-units\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.528782 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aaef9236-1230-4c5a-9f06-08458e473e03-ovnkube-script-lib\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528815 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c14334f4-fee6-411c-a8a4-228a183a7da2-cni-binary-copy\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528840 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528865 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528891 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90846c11-f046-48a5-a8a0-de0e57db04ee-konnectivity-ca\") pod \"konnectivity-agent-l6k5h\" (UID: \"90846c11-f046-48a5-a8a0-de0e57db04ee\") " pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528943 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-socket-dir-parent\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528968 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.528995 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-etc-openvswitch\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529047 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 15 07:25:10.529060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lctfv\" (UniqueName: \"kubernetes.io/projected/aaef9236-1230-4c5a-9f06-08458e473e03-kube-api-access-lctfv\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529106 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-run-netns\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjh2p\" (UniqueName: \"kubernetes.io/projected/c14334f4-fee6-411c-a8a4-228a183a7da2-kube-api-access-fjh2p\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529160 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g52j4\" (UniqueName: \"kubernetes.io/projected/5b7c5265-0d16-4645-8524-5c3ba1839641-kube-api-access-g52j4\") pod \"node-resolver-96dhs\" (UID: \"5b7c5265-0d16-4645-8524-5c3ba1839641\") " pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529184 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-device-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529209 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529232 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-hostroot\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529237 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-g6j7t\"" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20607e49-0155-468e-b8d4-0fda61b3be41-host-slash\") pod \"iptables-alerter-qmdkl\" (UID: \"20607e49-0155-468e-b8d4-0fda61b3be41\") " pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529292 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529349 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7c5265-0d16-4645-8524-5c3ba1839641-tmp-dir\") pod \"node-resolver-96dhs\" (UID: \"5b7c5265-0d16-4645-8524-5c3ba1839641\") " pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-slash\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529468 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-cni-bin\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529550 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-run-k8s-cni-cncf-io\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529589 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-var-lib-kubelet\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529600 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529613 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529618 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-run-multus-certs\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529671 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-daemon-config\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.529722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529698 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-node-log\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529723 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-log-socket\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529761 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-kubelet\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-run-systemd\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529821 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-run-openvswitch\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529847 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-run-ovn\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529864 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-cni-netd\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-var-lib-cni-multus\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529936 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aaef9236-1230-4c5a-9f06-08458e473e03-env-overrides\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e623486b-2e21-4cfc-844d-25fefc0f4418-dbus\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.529978 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-cni-dir\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530002 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aaef9236-1230-4c5a-9f06-08458e473e03-ovnkube-config\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530025 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-system-cni-dir\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530049 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90846c11-f046-48a5-a8a0-de0e57db04ee-agent-certs\") pod \"konnectivity-agent-l6k5h\" (UID: \"90846c11-f046-48a5-a8a0-de0e57db04ee\") " pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5b7c5265-0d16-4645-8524-5c3ba1839641-hosts-file\") pod \"node-resolver-96dhs\" (UID: \"5b7c5265-0d16-4645-8524-5c3ba1839641\") " pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530096 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e623486b-2e21-4cfc-844d-25fefc0f4418-kubelet-config\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530122 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-socket-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.530792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbmd9\" (UniqueName: \"kubernetes.io/projected/20607e49-0155-468e-b8d4-0fda61b3be41-kube-api-access-rbmd9\") pod \"iptables-alerter-qmdkl\" (UID: \"20607e49-0155-468e-b8d4-0fda61b3be41\") " pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.531849 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530171 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-var-lib-cni-bin\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.531849 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530195 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-etc-kubernetes\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.531849 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-sys-fs\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.531849 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-run-ovn-kubernetes\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.531849 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530365 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aaef9236-1230-4c5a-9f06-08458e473e03-ovn-node-metrics-cert\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.531849 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.530384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-cnibin\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.547826 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.547808 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wk7ns" Sep 15 07:25:10.555851 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.555827 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wk7ns" Sep 15 07:25:10.615874 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.615848 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 15 07:25:10.631447 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7c5265-0d16-4645-8524-5c3ba1839641-tmp-dir\") pod \"node-resolver-96dhs\" (UID: \"5b7c5265-0d16-4645-8524-5c3ba1839641\") " pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.631587 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631468 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-var-lib-kubelet\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.631587 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-host\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.631587 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-slash\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631595 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-cni-bin\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-run-k8s-cni-cncf-io\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-var-lib-kubelet\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631669 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-run-multus-certs\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631680 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-cni-bin\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631654 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-slash\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n8wt\" (UniqueName: \"kubernetes.io/projected/7ef119f1-8070-4d2a-b511-1fd19295935b-kube-api-access-6n8wt\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-daemon-config\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631751 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-run-k8s-cni-cncf-io\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-var-lib-kubelet\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.631764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631767 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-modprobe-d\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631790 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-run\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631805 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7c5265-0d16-4645-8524-5c3ba1839641-tmp-dir\") pod \"node-resolver-96dhs\" (UID: \"5b7c5265-0d16-4645-8524-5c3ba1839641\") " pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631812 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-run-multus-certs\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631829 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4c07647-b791-48b8-bd5e-01de110142d2-system-cni-dir\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-node-log\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631889 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-log-socket\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631900 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-node-log\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631909 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-kubelet\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631925 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-run-systemd\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631942 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-run-openvswitch\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631952 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-log-socket\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-run-ovn\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.631998 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-run-systemd\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632007 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-cni-netd\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632013 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-kubelet\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632043 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-run-openvswitch\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632074 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-cni-netd\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.632336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632073 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-run-ovn\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632047 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-var-lib-cni-multus\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632100 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-var-lib-cni-multus\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632433 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-daemon-config\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632136 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aaef9236-1230-4c5a-9f06-08458e473e03-env-overrides\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632631 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aaef9236-1230-4c5a-9f06-08458e473e03-env-overrides\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e623486b-2e21-4cfc-844d-25fefc0f4418-dbus\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-lib-modules\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632719 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khg64\" (UniqueName: \"kubernetes.io/projected/3e78afdb-75af-4944-a28b-7850f145d165-kube-api-access-khg64\") pod \"node-ca-k8pc6\" (UID: \"3e78afdb-75af-4944-a28b-7850f145d165\") " pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4c07647-b791-48b8-bd5e-01de110142d2-cnibin\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632772 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4c07647-b791-48b8-bd5e-01de110142d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-cni-dir\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aaef9236-1230-4c5a-9f06-08458e473e03-ovnkube-config\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632869 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-system-cni-dir\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632874 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-cni-dir\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632886 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90846c11-f046-48a5-a8a0-de0e57db04ee-agent-certs\") pod \"konnectivity-agent-l6k5h\" (UID: \"90846c11-f046-48a5-a8a0-de0e57db04ee\") " pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632843 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e623486b-2e21-4cfc-844d-25fefc0f4418-dbus\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:10.633247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632908 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5b7c5265-0d16-4645-8524-5c3ba1839641-hosts-file\") pod \"node-resolver-96dhs\" (UID: \"5b7c5265-0d16-4645-8524-5c3ba1839641\") " pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e623486b-2e21-4cfc-844d-25fefc0f4418-kubelet-config\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632941 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-socket-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbmd9\" (UniqueName: \"kubernetes.io/projected/20607e49-0155-468e-b8d4-0fda61b3be41-kube-api-access-rbmd9\") pod \"iptables-alerter-qmdkl\" (UID: \"20607e49-0155-468e-b8d4-0fda61b3be41\") " pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632972 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-var-lib-cni-bin\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633004 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-system-cni-dir\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633059 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5b7c5265-0d16-4645-8524-5c3ba1839641-hosts-file\") pod \"node-resolver-96dhs\" (UID: \"5b7c5265-0d16-4645-8524-5c3ba1839641\") " pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633088 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-etc-kubernetes\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.632995 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-etc-kubernetes\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633133 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-sys-fs\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e78afdb-75af-4944-a28b-7850f145d165-host\") pod \"node-ca-k8pc6\" (UID: \"3e78afdb-75af-4944-a28b-7850f145d165\") " pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633179 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-socket-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633190 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-run-ovn-kubernetes\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633218 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aaef9236-1230-4c5a-9f06-08458e473e03-ovn-node-metrics-cert\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-cnibin\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633313 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aaef9236-1230-4c5a-9f06-08458e473e03-ovnkube-config\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633311 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74lm\" (UniqueName: \"kubernetes.io/projected/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-kube-api-access-w74lm\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.634050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633106 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e623486b-2e21-4cfc-844d-25fefc0f4418-kubelet-config\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633360 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-kubernetes\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633383 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-sys\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633399 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-sys-fs\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-var-lib-openvswitch\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-conf-dir\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633450 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-var-lib-cni-bin\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-registration-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633519 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-run-ovn-kubernetes\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20607e49-0155-468e-b8d4-0fda61b3be41-iptables-alerter-script\") pod \"iptables-alerter-qmdkl\" (UID: \"20607e49-0155-468e-b8d4-0fda61b3be41\") " pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633583 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-var-lib-openvswitch\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633573 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-sysctl-d\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633635 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-sysctl-conf\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633639 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633658 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-systemd\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633682 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633740 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-run-netns\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.634937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e78afdb-75af-4944-a28b-7850f145d165-serviceca\") pod \"node-ca-k8pc6\" (UID: \"3e78afdb-75af-4944-a28b-7850f145d165\") " pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633843 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4c07647-b791-48b8-bd5e-01de110142d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633870 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-conf-dir\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-systemd-units\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aaef9236-1230-4c5a-9f06-08458e473e03-ovnkube-script-lib\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633933 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-registration-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c14334f4-fee6-411c-a8a4-228a183a7da2-cni-binary-copy\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633474 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-cnibin\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.633974 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-run-netns\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634024 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-systemd-units\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634014 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634212 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634252 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634291 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90846c11-f046-48a5-a8a0-de0e57db04ee-konnectivity-ca\") pod \"konnectivity-agent-l6k5h\" (UID: \"90846c11-f046-48a5-a8a0-de0e57db04ee\") " pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634318 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61db389e-742a-4164-9417-d95ca886cc78-etc-tuned\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634392 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4c07647-b791-48b8-bd5e-01de110142d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.635641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634418 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-socket-dir-parent\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634445 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634506 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wc8j\" (UniqueName: \"kubernetes.io/projected/f4c07647-b791-48b8-bd5e-01de110142d2-kube-api-access-2wc8j\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634559 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-etc-openvswitch\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lctfv\" (UniqueName: \"kubernetes.io/projected/aaef9236-1230-4c5a-9f06-08458e473e03-kube-api-access-lctfv\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634593 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c14334f4-fee6-411c-a8a4-228a183a7da2-cni-binary-copy\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634612 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-run-netns\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634619 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aaef9236-1230-4c5a-9f06-08458e473e03-ovnkube-script-lib\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634587 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20607e49-0155-468e-b8d4-0fda61b3be41-iptables-alerter-script\") pod \"iptables-alerter-qmdkl\" (UID: \"20607e49-0155-468e-b8d4-0fda61b3be41\") " pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.634664 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634665 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634720 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634745 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-multus-socket-dir-parent\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjh2p\" (UniqueName: \"kubernetes.io/projected/c14334f4-fee6-411c-a8a4-228a183a7da2-kube-api-access-fjh2p\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634770 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-host-run-netns\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634782 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aaef9236-1230-4c5a-9f06-08458e473e03-etc-openvswitch\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.634793 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret podName:e623486b-2e21-4cfc-844d-25fefc0f4418 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:11.134770322 +0000 UTC m=+2.957241735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret") pod "global-pull-secret-syncer-v8tt9" (UID: "e623486b-2e21-4cfc-844d-25fefc0f4418") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:10.636263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634895 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g52j4\" (UniqueName: \"kubernetes.io/projected/5b7c5265-0d16-4645-8524-5c3ba1839641-kube-api-access-g52j4\") pod \"node-resolver-96dhs\" (UID: \"5b7c5265-0d16-4645-8524-5c3ba1839641\") " pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634921 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-device-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634947 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.634992 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-hostroot\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.635021 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61db389e-742a-4164-9417-d95ca886cc78-tmp\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.635033 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-device-dir\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.635072 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c14334f4-fee6-411c-a8a4-228a183a7da2-hostroot\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.635152 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4c07647-b791-48b8-bd5e-01de110142d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.635186 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20607e49-0155-468e-b8d4-0fda61b3be41-host-slash\") pod \"iptables-alerter-qmdkl\" (UID: \"20607e49-0155-468e-b8d4-0fda61b3be41\") " pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.635214 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-sysconfig\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.635241 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6khfk\" (UniqueName: \"kubernetes.io/projected/61db389e-742a-4164-9417-d95ca886cc78-kube-api-access-6khfk\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.635285 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20607e49-0155-468e-b8d4-0fda61b3be41-host-slash\") pod \"iptables-alerter-qmdkl\" (UID: \"20607e49-0155-468e-b8d4-0fda61b3be41\") " pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.637064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.635240 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90846c11-f046-48a5-a8a0-de0e57db04ee-konnectivity-ca\") pod \"konnectivity-agent-l6k5h\" (UID: \"90846c11-f046-48a5-a8a0-de0e57db04ee\") " pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:10.638570 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.638544 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aaef9236-1230-4c5a-9f06-08458e473e03-ovn-node-metrics-cert\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.638686 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.638584 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90846c11-f046-48a5-a8a0-de0e57db04ee-agent-certs\") pod \"konnectivity-agent-l6k5h\" (UID: \"90846c11-f046-48a5-a8a0-de0e57db04ee\") " pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:10.642799 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.642774 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbmd9\" (UniqueName: \"kubernetes.io/projected/20607e49-0155-468e-b8d4-0fda61b3be41-kube-api-access-rbmd9\") pod \"iptables-alerter-qmdkl\" (UID: \"20607e49-0155-468e-b8d4-0fda61b3be41\") " pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.643352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.643324 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w74lm\" (UniqueName: \"kubernetes.io/projected/98080ddc-50f4-48a0-a0ef-59f24adf7ea1-kube-api-access-w74lm\") pod \"aws-ebs-csi-driver-node-kkrst\" (UID: \"98080ddc-50f4-48a0-a0ef-59f24adf7ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.648059 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.646370 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:25:10.648059 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.646395 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:25:10.648059 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.646409 2577 projected.go:196] Error preparing data for projected volume kube-api-access-g4b7g for pod openshift-network-diagnostics/network-check-target-m5skf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:10.648059 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.646478 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g podName:735140f1-e909-449e-8129-be3752b71965 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:11.146460609 +0000 UTC m=+2.968932015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g4b7g" (UniqueName: "kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g") pod "network-check-target-m5skf" (UID: "735140f1-e909-449e-8129-be3752b71965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:10.648059 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.647609 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lctfv\" (UniqueName: \"kubernetes.io/projected/aaef9236-1230-4c5a-9f06-08458e473e03-kube-api-access-lctfv\") pod \"ovnkube-node-4k5xc\" (UID: \"aaef9236-1230-4c5a-9f06-08458e473e03\") " pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.648059 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.647791 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:25:10.648977 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.648490 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g52j4\" (UniqueName: \"kubernetes.io/projected/5b7c5265-0d16-4645-8524-5c3ba1839641-kube-api-access-g52j4\") pod \"node-resolver-96dhs\" (UID: \"5b7c5265-0d16-4645-8524-5c3ba1839641\") " pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.648977 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.648550 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjh2p\" (UniqueName: \"kubernetes.io/projected/c14334f4-fee6-411c-a8a4-228a183a7da2-kube-api-access-fjh2p\") pod \"multus-2vds4\" (UID: \"c14334f4-fee6-411c-a8a4-228a183a7da2\") " pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.689403 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.689345 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" event={"ID":"b9dc28918bbb8a1972a191b9a118ed97","Type":"ContainerStarted","Data":"09861d3a1933139d726a82b511347452733c1d423d556362dd72d809ca6af47d"} Sep 15 07:25:10.690639 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.690617 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal" event={"ID":"1bc2623acdeb0a77dc30a9def2462fc4","Type":"ContainerStarted","Data":"0a64710a1539198bcf3ff32c433549cb8c8904c4f9c97a62bdcbc0fbb0b62178"} Sep 15 07:25:10.736470 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736442 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-kubernetes\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736615 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736478 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-sys\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736615 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736519 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-sysctl-d\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736615 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-sysctl-conf\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736615 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736588 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-kubernetes\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736615 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736600 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-sys\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736594 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-systemd\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736651 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-systemd\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736653 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736715 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e78afdb-75af-4944-a28b-7850f145d165-serviceca\") pod \"node-ca-k8pc6\" (UID: \"3e78afdb-75af-4944-a28b-7850f145d165\") " pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4c07647-b791-48b8-bd5e-01de110142d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.736743 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736751 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-sysctl-conf\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736748 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-sysctl-d\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736767 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61db389e-742a-4164-9417-d95ca886cc78-etc-tuned\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:10.736796 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs podName:7ef119f1-8070-4d2a-b511-1fd19295935b nodeName:}" failed. No retries permitted until 2026-09-15 07:25:11.236777244 +0000 UTC m=+3.059248629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs") pod "network-metrics-daemon-f9mmh" (UID: "7ef119f1-8070-4d2a-b511-1fd19295935b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:10.736872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.736818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4c07647-b791-48b8-bd5e-01de110142d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737085 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wc8j\" (UniqueName: \"kubernetes.io/projected/f4c07647-b791-48b8-bd5e-01de110142d2-kube-api-access-2wc8j\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737137 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61db389e-742a-4164-9417-d95ca886cc78-tmp\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737166 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4c07647-b791-48b8-bd5e-01de110142d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-sysconfig\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737218 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6khfk\" (UniqueName: \"kubernetes.io/projected/61db389e-742a-4164-9417-d95ca886cc78-kube-api-access-6khfk\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737262 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-var-lib-kubelet\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737273 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e78afdb-75af-4944-a28b-7850f145d165-serviceca\") pod \"node-ca-k8pc6\" (UID: \"3e78afdb-75af-4944-a28b-7850f145d165\") " pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737289 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-host\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737329 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n8wt\" (UniqueName: \"kubernetes.io/projected/7ef119f1-8070-4d2a-b511-1fd19295935b-kube-api-access-6n8wt\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737347 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-sysconfig\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737356 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-modprobe-d\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737361 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4c07647-b791-48b8-bd5e-01de110142d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.737420 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737381 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-run\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737494 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-etc-modprobe-d\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737628 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-run\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737638 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4c07647-b791-48b8-bd5e-01de110142d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737656 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-var-lib-kubelet\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737667 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4c07647-b791-48b8-bd5e-01de110142d2-system-cni-dir\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737704 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-host\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737747 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4c07647-b791-48b8-bd5e-01de110142d2-system-cni-dir\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737830 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-lib-modules\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khg64\" (UniqueName: \"kubernetes.io/projected/3e78afdb-75af-4944-a28b-7850f145d165-kube-api-access-khg64\") pod \"node-ca-k8pc6\" (UID: \"3e78afdb-75af-4944-a28b-7850f145d165\") " pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737904 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4c07647-b791-48b8-bd5e-01de110142d2-cnibin\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737938 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4c07647-b791-48b8-bd5e-01de110142d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.737977 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e78afdb-75af-4944-a28b-7850f145d165-host\") pod \"node-ca-k8pc6\" (UID: \"3e78afdb-75af-4944-a28b-7850f145d165\") " pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.738007 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4c07647-b791-48b8-bd5e-01de110142d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.738024 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4c07647-b791-48b8-bd5e-01de110142d2-cnibin\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.738106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.738047 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e78afdb-75af-4944-a28b-7850f145d165-host\") pod \"node-ca-k8pc6\" (UID: \"3e78afdb-75af-4944-a28b-7850f145d165\") " pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.738697 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.738146 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61db389e-742a-4164-9417-d95ca886cc78-lib-modules\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.738697 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.738448 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4c07647-b791-48b8-bd5e-01de110142d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.739356 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.739336 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61db389e-742a-4164-9417-d95ca886cc78-etc-tuned\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.740381 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.740363 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61db389e-742a-4164-9417-d95ca886cc78-tmp\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.747986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.747928 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wc8j\" (UniqueName: \"kubernetes.io/projected/f4c07647-b791-48b8-bd5e-01de110142d2-kube-api-access-2wc8j\") pod \"multus-additional-cni-plugins-22spd\" (UID: \"f4c07647-b791-48b8-bd5e-01de110142d2\") " pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.747986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.747932 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6khfk\" (UniqueName: \"kubernetes.io/projected/61db389e-742a-4164-9417-d95ca886cc78-kube-api-access-6khfk\") pod \"tuned-8zr7v\" (UID: \"61db389e-742a-4164-9417-d95ca886cc78\") " pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.748319 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.748301 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khg64\" (UniqueName: \"kubernetes.io/projected/3e78afdb-75af-4944-a28b-7850f145d165-kube-api-access-khg64\") pod \"node-ca-k8pc6\" (UID: \"3e78afdb-75af-4944-a28b-7850f145d165\") " pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.748474 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.748453 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n8wt\" (UniqueName: \"kubernetes.io/projected/7ef119f1-8070-4d2a-b511-1fd19295935b-kube-api-access-6n8wt\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:10.819167 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.819131 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:10.826495 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.826471 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-96dhs" Sep 15 07:25:10.831219 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:10.831195 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90846c11_f046_48a5_a8a0_de0e57db04ee.slice/crio-a7e66578ae31131c1db5211ef6d1dd3393339bba636310e10897363aff2b0aad WatchSource:0}: Error finding container a7e66578ae31131c1db5211ef6d1dd3393339bba636310e10897363aff2b0aad: Status 404 returned error can't find the container with id a7e66578ae31131c1db5211ef6d1dd3393339bba636310e10897363aff2b0aad Sep 15 07:25:10.835857 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:10.835835 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7c5265_0d16_4645_8524_5c3ba1839641.slice/crio-2ee09d6a5a5effd6a690396cd44b4dbe6010b9b8cfe08823007d0dfd40aa54ae WatchSource:0}: Error finding container 2ee09d6a5a5effd6a690396cd44b4dbe6010b9b8cfe08823007d0dfd40aa54ae: Status 404 returned error can't find the container with id 2ee09d6a5a5effd6a690396cd44b4dbe6010b9b8cfe08823007d0dfd40aa54ae Sep 15 07:25:10.837747 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.837726 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qmdkl" Sep 15 07:25:10.843872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.843849 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k8pc6" Sep 15 07:25:10.846630 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:10.846581 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20607e49_0155_468e_b8d4_0fda61b3be41.slice/crio-0aeffe95404ae8e9defcc76c9db1a6b92965936fcf54d7abb36ddacb01a62e7f WatchSource:0}: Error finding container 0aeffe95404ae8e9defcc76c9db1a6b92965936fcf54d7abb36ddacb01a62e7f: Status 404 returned error can't find the container with id 0aeffe95404ae8e9defcc76c9db1a6b92965936fcf54d7abb36ddacb01a62e7f Sep 15 07:25:10.849402 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.849384 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:10.852020 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:10.851996 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e78afdb_75af_4944_a28b_7850f145d165.slice/crio-f6a7b0e71244432972840920050f55f89f06ce9f6a7f359abd05e27867e5372c WatchSource:0}: Error finding container f6a7b0e71244432972840920050f55f89f06ce9f6a7f359abd05e27867e5372c: Status 404 returned error can't find the container with id f6a7b0e71244432972840920050f55f89f06ce9f6a7f359abd05e27867e5372c Sep 15 07:25:10.856987 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.856966 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" Sep 15 07:25:10.858317 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:10.858294 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaef9236_1230_4c5a_9f06_08458e473e03.slice/crio-02c851790e05819ad7d8a28a0266a7db3d6fbfc25c5dc308f3f52b0894a82020 WatchSource:0}: Error finding container 02c851790e05819ad7d8a28a0266a7db3d6fbfc25c5dc308f3f52b0894a82020: Status 404 returned error can't find the container with id 02c851790e05819ad7d8a28a0266a7db3d6fbfc25c5dc308f3f52b0894a82020 Sep 15 07:25:10.864682 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.864663 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2vds4" Sep 15 07:25:10.865724 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:10.865702 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98080ddc_50f4_48a0_a0ef_59f24adf7ea1.slice/crio-e6025a73db752ea6995e99911918d1047c12c923ae2e4a1133854fc021db5a36 WatchSource:0}: Error finding container e6025a73db752ea6995e99911918d1047c12c923ae2e4a1133854fc021db5a36: Status 404 returned error can't find the container with id e6025a73db752ea6995e99911918d1047c12c923ae2e4a1133854fc021db5a36 Sep 15 07:25:10.870771 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.870748 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" Sep 15 07:25:10.872369 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:10.872347 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc14334f4_fee6_411c_a8a4_228a183a7da2.slice/crio-ce1a0446903d9f9cd1455a78414261258c6f18ad4685d1e6beb449483527361e WatchSource:0}: Error finding container ce1a0446903d9f9cd1455a78414261258c6f18ad4685d1e6beb449483527361e: Status 404 returned error can't find the container with id ce1a0446903d9f9cd1455a78414261258c6f18ad4685d1e6beb449483527361e Sep 15 07:25:10.877065 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:10.877045 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-22spd" Sep 15 07:25:10.887641 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:25:10.887616 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c07647_b791_48b8_bd5e_01de110142d2.slice/crio-8b46d14b5310e6e4fb1e8d271e7a20fe93a87cd9ed6df816ec555d1c5e2e3409 WatchSource:0}: Error finding container 8b46d14b5310e6e4fb1e8d271e7a20fe93a87cd9ed6df816ec555d1c5e2e3409: Status 404 returned error can't find the container with id 8b46d14b5310e6e4fb1e8d271e7a20fe93a87cd9ed6df816ec555d1c5e2e3409 Sep 15 07:25:11.141757 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.141668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:11.141917 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.141815 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:11.141917 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.141870 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret podName:e623486b-2e21-4cfc-844d-25fefc0f4418 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:12.14185495 +0000 UTC m=+3.964326340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret") pod "global-pull-secret-syncer-v8tt9" (UID: "e623486b-2e21-4cfc-844d-25fefc0f4418") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:11.242139 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.242092 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:11.242327 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.242188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:11.242327 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.242297 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:11.242327 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.242296 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:25:11.242327 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.242326 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:25:11.242581 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.242340 2577 projected.go:196] Error preparing data for projected volume kube-api-access-g4b7g for pod openshift-network-diagnostics/network-check-target-m5skf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:11.242581 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.242366 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs podName:7ef119f1-8070-4d2a-b511-1fd19295935b nodeName:}" failed. No retries permitted until 2026-09-15 07:25:12.242336145 +0000 UTC m=+4.064807544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs") pod "network-metrics-daemon-f9mmh" (UID: "7ef119f1-8070-4d2a-b511-1fd19295935b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:11.242581 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.242396 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g podName:735140f1-e909-449e-8129-be3752b71965 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:12.24237723 +0000 UTC m=+4.064848621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4b7g" (UniqueName: "kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g") pod "network-check-target-m5skf" (UID: "735140f1-e909-449e-8129-be3752b71965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:11.558648 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.558500 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 07:20:11 +0000 UTC" deadline="2028-02-29 03:36:16.254755921 +0000 UTC" Sep 15 07:25:11.558648 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.558560 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12764h11m4.696200442s" Sep 15 07:25:11.684986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.684950 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:11.685218 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.685193 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:11.685715 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.685648 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:11.685825 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:11.685750 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:11.697661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.697630 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22spd" event={"ID":"f4c07647-b791-48b8-bd5e-01de110142d2","Type":"ContainerStarted","Data":"8b46d14b5310e6e4fb1e8d271e7a20fe93a87cd9ed6df816ec555d1c5e2e3409"} Sep 15 07:25:11.701927 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.701901 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" event={"ID":"98080ddc-50f4-48a0-a0ef-59f24adf7ea1","Type":"ContainerStarted","Data":"e6025a73db752ea6995e99911918d1047c12c923ae2e4a1133854fc021db5a36"} Sep 15 07:25:11.706452 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.706424 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" event={"ID":"aaef9236-1230-4c5a-9f06-08458e473e03","Type":"ContainerStarted","Data":"02c851790e05819ad7d8a28a0266a7db3d6fbfc25c5dc308f3f52b0894a82020"} Sep 15 07:25:11.709363 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.709340 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:25:11.709862 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.709834 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qmdkl" event={"ID":"20607e49-0155-468e-b8d4-0fda61b3be41","Type":"ContainerStarted","Data":"0aeffe95404ae8e9defcc76c9db1a6b92965936fcf54d7abb36ddacb01a62e7f"} Sep 15 07:25:11.712056 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.712027 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-96dhs" event={"ID":"5b7c5265-0d16-4645-8524-5c3ba1839641","Type":"ContainerStarted","Data":"2ee09d6a5a5effd6a690396cd44b4dbe6010b9b8cfe08823007d0dfd40aa54ae"} Sep 15 07:25:11.716725 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.716693 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l6k5h" event={"ID":"90846c11-f046-48a5-a8a0-de0e57db04ee","Type":"ContainerStarted","Data":"a7e66578ae31131c1db5211ef6d1dd3393339bba636310e10897363aff2b0aad"} Sep 15 07:25:11.719423 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.719396 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" event={"ID":"61db389e-742a-4164-9417-d95ca886cc78","Type":"ContainerStarted","Data":"0eaffa0f27975c9b8cdbeac1136fb351e50d7d75ce89543ac8af689bf59c4aa7"} Sep 15 07:25:11.721849 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.721823 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2vds4" event={"ID":"c14334f4-fee6-411c-a8a4-228a183a7da2","Type":"ContainerStarted","Data":"ce1a0446903d9f9cd1455a78414261258c6f18ad4685d1e6beb449483527361e"} Sep 15 07:25:11.725027 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:11.725003 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k8pc6" event={"ID":"3e78afdb-75af-4944-a28b-7850f145d165","Type":"ContainerStarted","Data":"f6a7b0e71244432972840920050f55f89f06ce9f6a7f359abd05e27867e5372c"} Sep 15 07:25:12.149890 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:12.149852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:12.150097 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:12.150036 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:12.150173 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:12.150117 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret podName:e623486b-2e21-4cfc-844d-25fefc0f4418 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:14.150094802 +0000 UTC m=+5.972566187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret") pod "global-pull-secret-syncer-v8tt9" (UID: "e623486b-2e21-4cfc-844d-25fefc0f4418") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:12.251184 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:12.250283 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:12.251184 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:12.250360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:12.251184 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:12.250560 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:25:12.251184 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:12.250585 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:25:12.251184 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:12.250600 2577 projected.go:196] Error preparing data for projected volume kube-api-access-g4b7g for pod openshift-network-diagnostics/network-check-target-m5skf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:12.251184 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:12.250664 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g podName:735140f1-e909-449e-8129-be3752b71965 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:14.250644887 +0000 UTC m=+6.073116289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4b7g" (UniqueName: "kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g") pod "network-check-target-m5skf" (UID: "735140f1-e909-449e-8129-be3752b71965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:12.251184 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:12.251086 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:12.251184 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:12.251142 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs podName:7ef119f1-8070-4d2a-b511-1fd19295935b nodeName:}" failed. No retries permitted until 2026-09-15 07:25:14.251125454 +0000 UTC m=+6.073596854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs") pod "network-metrics-daemon-f9mmh" (UID: "7ef119f1-8070-4d2a-b511-1fd19295935b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:12.559528 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:12.559410 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 07:20:11 +0000 UTC" deadline="2028-03-29 21:08:41.649459811 +0000 UTC" Sep 15 07:25:12.559528 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:12.559457 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13477h43m29.09000819s" Sep 15 07:25:12.687676 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:12.685297 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:12.687676 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:12.685446 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:13.684369 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:13.684317 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:13.685019 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:13.684458 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:13.685019 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:13.684649 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:13.685019 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:13.684778 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:13.761990 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:13.761948 2577 generic.go:356] "Generic (PLEG): container finished" podID="b9dc28918bbb8a1972a191b9a118ed97" containerID="92a5b3993ab99cc0aec6cd19480a5becf1b1419499d9148b8685b639df6cbbb3" exitCode=0 Sep 15 07:25:13.762180 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:13.762010 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" event={"ID":"b9dc28918bbb8a1972a191b9a118ed97","Type":"ContainerDied","Data":"92a5b3993ab99cc0aec6cd19480a5becf1b1419499d9148b8685b639df6cbbb3"} Sep 15 07:25:14.166972 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:14.166929 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:14.167149 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:14.167097 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:14.167233 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:14.167167 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret podName:e623486b-2e21-4cfc-844d-25fefc0f4418 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:18.167146632 +0000 UTC m=+9.989618041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret") pod "global-pull-secret-syncer-v8tt9" (UID: "e623486b-2e21-4cfc-844d-25fefc0f4418") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:14.267526 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:14.267470 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:14.267711 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:14.267573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:14.267711 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:14.267704 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:14.267809 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:14.267772 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs podName:7ef119f1-8070-4d2a-b511-1fd19295935b nodeName:}" failed. No retries permitted until 2026-09-15 07:25:18.267752191 +0000 UTC m=+10.090223581 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs") pod "network-metrics-daemon-f9mmh" (UID: "7ef119f1-8070-4d2a-b511-1fd19295935b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:14.268017 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:14.267977 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:25:14.268017 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:14.268002 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:25:14.268017 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:14.268016 2577 projected.go:196] Error preparing data for projected volume kube-api-access-g4b7g for pod openshift-network-diagnostics/network-check-target-m5skf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:14.268186 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:14.268061 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g podName:735140f1-e909-449e-8129-be3752b71965 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:18.268046492 +0000 UTC m=+10.090517878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4b7g" (UniqueName: "kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g") pod "network-check-target-m5skf" (UID: "735140f1-e909-449e-8129-be3752b71965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:14.686001 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:14.685443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:14.686001 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:14.685609 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:15.685217 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:15.685179 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:15.685431 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:15.685324 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:15.685431 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:15.685403 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:15.685768 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:15.685480 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:16.685231 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:16.685189 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:16.686037 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:16.685371 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:17.685280 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:17.685199 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:17.685721 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:17.685333 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:17.685721 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:17.685384 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:17.685721 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:17.685514 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:18.201828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:18.201786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:18.202040 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:18.201958 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:18.202040 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:18.202018 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret podName:e623486b-2e21-4cfc-844d-25fefc0f4418 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:26.20200091 +0000 UTC m=+18.024472317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret") pod "global-pull-secret-syncer-v8tt9" (UID: "e623486b-2e21-4cfc-844d-25fefc0f4418") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:18.302411 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:18.302367 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:18.302581 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:18.302466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:18.302749 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:18.302717 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:18.302820 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:18.302789 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs podName:7ef119f1-8070-4d2a-b511-1fd19295935b nodeName:}" failed. No retries permitted until 2026-09-15 07:25:26.302769287 +0000 UTC m=+18.125240673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs") pod "network-metrics-daemon-f9mmh" (UID: "7ef119f1-8070-4d2a-b511-1fd19295935b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:18.312694 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:18.312665 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:25:18.312827 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:18.312705 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:25:18.312827 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:18.312719 2577 projected.go:196] Error preparing data for projected volume kube-api-access-g4b7g for pod openshift-network-diagnostics/network-check-target-m5skf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:18.312827 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:18.312787 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g podName:735140f1-e909-449e-8129-be3752b71965 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:26.312760174 +0000 UTC m=+18.135231573 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4b7g" (UniqueName: "kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g") pod "network-check-target-m5skf" (UID: "735140f1-e909-449e-8129-be3752b71965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:18.686581 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:18.686291 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:18.687008 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:18.686638 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:19.685649 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:19.685222 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:19.685649 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:19.685318 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:19.685649 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:19.685536 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:19.685649 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:19.685648 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:20.684936 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:20.684899 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:20.685389 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:20.685055 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:21.685149 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:21.685112 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:21.685724 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:21.685241 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:21.685724 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:21.685401 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:21.685724 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:21.685520 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:22.684476 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:22.684450 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:22.684616 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:22.684596 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:23.684351 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:23.684322 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:23.684351 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:23.684345 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:23.684868 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:23.684435 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:23.684868 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:23.684580 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:24.685313 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:24.685279 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:24.685747 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:24.685420 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:25.684645 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:25.684610 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:25.684883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:25.684610 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:25.684883 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:25.684741 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:25.684883 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:25.684790 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:26.259179 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:26.259136 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:26.259623 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:26.259315 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:26.259623 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:26.259394 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret podName:e623486b-2e21-4cfc-844d-25fefc0f4418 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:42.259372865 +0000 UTC m=+34.081844270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret") pod "global-pull-secret-syncer-v8tt9" (UID: "e623486b-2e21-4cfc-844d-25fefc0f4418") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:26.360316 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:26.360273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:26.360502 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:26.360334 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:26.360502 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:26.360443 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:26.360502 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:26.360477 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:25:26.360644 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:26.360508 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:25:26.360644 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:26.360517 2577 projected.go:196] Error preparing data for projected volume kube-api-access-g4b7g for pod openshift-network-diagnostics/network-check-target-m5skf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:26.360644 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:26.360522 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs podName:7ef119f1-8070-4d2a-b511-1fd19295935b nodeName:}" failed. No retries permitted until 2026-09-15 07:25:42.360506398 +0000 UTC m=+34.182977782 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs") pod "network-metrics-daemon-f9mmh" (UID: "7ef119f1-8070-4d2a-b511-1fd19295935b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:26.360644 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:26.360560 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g podName:735140f1-e909-449e-8129-be3752b71965 nodeName:}" failed. No retries permitted until 2026-09-15 07:25:42.360546189 +0000 UTC m=+34.183017574 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4b7g" (UniqueName: "kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g") pod "network-check-target-m5skf" (UID: "735140f1-e909-449e-8129-be3752b71965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:26.684425 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:26.684392 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:26.684600 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:26.684559 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:27.684414 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:27.684365 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:27.684414 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:27.684403 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:27.684875 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:27.684535 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:27.684875 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:27.684669 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:28.685858 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:28.685829 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:28.686310 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:28.685933 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:29.684716 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:29.684681 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:29.684886 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:29.684682 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:29.684886 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:29.684807 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:29.684886 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:29.684857 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:30.685606 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.685420 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:30.686234 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:30.685682 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:30.799606 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.799522 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal" event={"ID":"1bc2623acdeb0a77dc30a9def2462fc4","Type":"ContainerStarted","Data":"e2933c7b1d4469daad1bb84d154a0c6f89372d05d38b7e452e221db2b3cb2f1e"} Sep 15 07:25:30.801279 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.801249 2577 generic.go:356] "Generic (PLEG): container finished" podID="f4c07647-b791-48b8-bd5e-01de110142d2" containerID="f76fc5874122dfa6505edd856a5d9d046fe19cc6d1e7388c5524875a5638d5fb" exitCode=0 Sep 15 07:25:30.801417 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.801331 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22spd" event={"ID":"f4c07647-b791-48b8-bd5e-01de110142d2","Type":"ContainerDied","Data":"f76fc5874122dfa6505edd856a5d9d046fe19cc6d1e7388c5524875a5638d5fb"} Sep 15 07:25:30.803949 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.803926 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" event={"ID":"98080ddc-50f4-48a0-a0ef-59f24adf7ea1","Type":"ContainerStarted","Data":"4679b718b9c76360f822891acde0f82a685696863d916ea76d66abdea6850565"} Sep 15 07:25:30.808051 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.807990 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" event={"ID":"aaef9236-1230-4c5a-9f06-08458e473e03","Type":"ContainerStarted","Data":"5b8846a92cd21731b96391cd8e5385f250675ad41a82db5f44ee47402ffcaa6f"} Sep 15 07:25:30.808165 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.808064 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" event={"ID":"aaef9236-1230-4c5a-9f06-08458e473e03","Type":"ContainerStarted","Data":"5f985ab7c9ea1c8cfaaf8fe51b154f1c7f523c60185c04a211e16ff90c36d60e"} Sep 15 07:25:30.808165 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.808112 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" event={"ID":"aaef9236-1230-4c5a-9f06-08458e473e03","Type":"ContainerStarted","Data":"4491d094f176939153ebb303aa784c3ab37e9e3a53c6c4a456604e4de490a664"} Sep 15 07:25:30.808165 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.808159 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" event={"ID":"aaef9236-1230-4c5a-9f06-08458e473e03","Type":"ContainerStarted","Data":"e17f4d3d972ce3ea2a044fe25065db59c935be6e94613e03ee77a6d18158eaa9"} Sep 15 07:25:30.808318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.808207 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" event={"ID":"aaef9236-1230-4c5a-9f06-08458e473e03","Type":"ContainerStarted","Data":"39cec9e6a438c758e27e3585cf52c38c53304ecb60b2f3cd064efc9d869dce83"} Sep 15 07:25:30.808318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.808253 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" event={"ID":"aaef9236-1230-4c5a-9f06-08458e473e03","Type":"ContainerStarted","Data":"191ba1e813a0078512ffd582bde3428c19b2b75c9ea632ee8c411f86d33022ae"} Sep 15 07:25:30.809737 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.809712 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-96dhs" event={"ID":"5b7c5265-0d16-4645-8524-5c3ba1839641","Type":"ContainerStarted","Data":"fc8966cdb7e21f382cef8e51243e123540e9ffb872da2e871f2a6420c43c4cd6"} Sep 15 07:25:30.811506 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.811446 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l6k5h" event={"ID":"90846c11-f046-48a5-a8a0-de0e57db04ee","Type":"ContainerStarted","Data":"10bc829d4f9aa33e968e4344379fb83361baf9c5c38acbd9b269a38888900371"} Sep 15 07:25:30.813617 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.813585 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" event={"ID":"b9dc28918bbb8a1972a191b9a118ed97","Type":"ContainerStarted","Data":"660b5ea2b2128472fd146c216acbc1c39ccd87860c758db63c09c2a62e3053f3"} Sep 15 07:25:30.815668 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.815644 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" event={"ID":"61db389e-742a-4164-9417-d95ca886cc78","Type":"ContainerStarted","Data":"94a5358b9af4bae6c53b5953c7e1775a0998dbcf2cb7a94c21be415d898bc5ff"} Sep 15 07:25:30.817529 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.817507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2vds4" event={"ID":"c14334f4-fee6-411c-a8a4-228a183a7da2","Type":"ContainerStarted","Data":"e064f97b806394a470167de5f9d05a896940c110ea5dc05c05166d2e025a66c5"} Sep 15 07:25:30.819225 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.819202 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k8pc6" event={"ID":"3e78afdb-75af-4944-a28b-7850f145d165","Type":"ContainerStarted","Data":"ba4120f6db7e95b2630383c433606c14a5431f3dc95d0bd0172849d42d216e99"} Sep 15 07:25:30.838794 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.838756 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-38.ec2.internal" podStartSLOduration=20.838740025 podStartE2EDuration="20.838740025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:30.819286259 +0000 UTC m=+22.641757665" watchObservedRunningTime="2026-09-15 07:25:30.838740025 +0000 UTC m=+22.661211433" Sep 15 07:25:30.839065 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.839030 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2vds4" podStartSLOduration=2.867343921 podStartE2EDuration="21.839023447s" totalImagesPullingTime="18.971679526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:30.83855143 +0000 UTC m=+22.661022836" watchObservedRunningTime="2026-09-15 07:25:30.839023447 +0000 UTC m=+22.661494854" Sep 15 07:25:30.871736 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.871694 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l6k5h" podStartSLOduration=2.916759673 podStartE2EDuration="21.871680949s" totalImagesPullingTime="18.954921276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:30.871614455 +0000 UTC m=+22.694085859" watchObservedRunningTime="2026-09-15 07:25:30.871680949 +0000 UTC m=+22.694152355" Sep 15 07:25:30.871912 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.871880 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8zr7v" podStartSLOduration=2.969044473 podStartE2EDuration="21.871873112s" totalImagesPullingTime="18.902828639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:30.857184354 +0000 UTC m=+22.679655751" watchObservedRunningTime="2026-09-15 07:25:30.871873112 +0000 UTC m=+22.694344519" Sep 15 07:25:30.908578 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.908543 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k8pc6" podStartSLOduration=2.97471776 podStartE2EDuration="21.908529833s" totalImagesPullingTime="18.933812073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:30.908301633 +0000 UTC m=+22.730773041" watchObservedRunningTime="2026-09-15 07:25:30.908529833 +0000 UTC m=+22.731001228" Sep 15 07:25:30.926966 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:30.926922 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-38.ec2.internal" podStartSLOduration=20.926910878 podStartE2EDuration="20.926910878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:30.926716766 +0000 UTC m=+22.749188170" watchObservedRunningTime="2026-09-15 07:25:30.926910878 +0000 UTC m=+22.749382284" Sep 15 07:25:31.256839 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.256819 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 15 07:25:31.576727 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.576602 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T07:25:31.256835165Z","UUID":"b119e0d9-32c8-4368-a0d6-0c38daee8474","Handler":null,"Name":"","Endpoint":""} Sep 15 07:25:31.577910 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.577892 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 15 07:25:31.578022 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.577921 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 15 07:25:31.684529 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.684498 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:31.684677 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.684535 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:31.684677 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:31.684648 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:31.684793 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:31.684768 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:31.824159 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.824120 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" event={"ID":"98080ddc-50f4-48a0-a0ef-59f24adf7ea1","Type":"ContainerStarted","Data":"cd8795dc69ab2e3e82a01c6b1a144e56e1bca45dec7f854639cd3149a5d628ca"} Sep 15 07:25:31.825924 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.825894 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qmdkl" event={"ID":"20607e49-0155-468e-b8d4-0fda61b3be41","Type":"ContainerStarted","Data":"5aae1f49ebf461be3fd2be8a30561c18932df89045e121a64736056af2cb715b"} Sep 15 07:25:31.843646 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.843575 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qmdkl" podStartSLOduration=3.8932676170000002 podStartE2EDuration="22.843554732s" totalImagesPullingTime="18.950287115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:31.843192249 +0000 UTC m=+23.665663654" watchObservedRunningTime="2026-09-15 07:25:31.843554732 +0000 UTC m=+23.666026139" Sep 15 07:25:31.844182 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:31.844151 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-96dhs" podStartSLOduration=3.893188481 podStartE2EDuration="22.844142342s" totalImagesPullingTime="18.950953861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:30.950846721 +0000 UTC m=+22.773318127" watchObservedRunningTime="2026-09-15 07:25:31.844142342 +0000 UTC m=+23.666613748" Sep 15 07:25:32.684428 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:32.684400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:32.684579 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:32.684555 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:32.830018 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:32.829937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" event={"ID":"98080ddc-50f4-48a0-a0ef-59f24adf7ea1","Type":"ContainerStarted","Data":"e9f111ef8d22d609617920ad2357ddec9ff3660c92fcfe91efae1742a833283c"} Sep 15 07:25:32.833191 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:32.833167 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" event={"ID":"aaef9236-1230-4c5a-9f06-08458e473e03","Type":"ContainerStarted","Data":"4d40a4c04ccf67ff2967b45943680143abfb5c3c7cb247cd19f0859234c7df95"} Sep 15 07:25:32.851431 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:32.851387 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkrst" podStartSLOduration=2.983573591 podStartE2EDuration="23.851375261s" totalImagesPullingTime="20.86780167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:32.85104265 +0000 UTC m=+24.673514056" watchObservedRunningTime="2026-09-15 07:25:32.851375261 +0000 UTC m=+24.673846666" Sep 15 07:25:33.685250 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:33.685085 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:33.685406 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:33.685092 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:33.685406 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:33.685317 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:33.685406 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:33.685392 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:34.684732 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:34.684702 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:34.685208 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:34.684840 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:34.844227 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:34.844185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" event={"ID":"aaef9236-1230-4c5a-9f06-08458e473e03","Type":"ContainerStarted","Data":"bdf4e760e494568fbc959bafc23b86a06ffffa90b488e9ac9eda9387eadce4bb"} Sep 15 07:25:34.844781 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:34.844755 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:34.844956 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:34.844836 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:34.844956 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:34.844887 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:34.862271 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:34.862164 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:34.863007 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:34.862852 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:25:34.874660 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:34.874298 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" podStartSLOduration=6.893377132 podStartE2EDuration="25.874281783s" totalImagesPullingTime="18.980904651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:34.87416997 +0000 UTC m=+26.696641379" watchObservedRunningTime="2026-09-15 07:25:34.874281783 +0000 UTC m=+26.696753189" Sep 15 07:25:35.129702 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:35.129627 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:35.130275 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:35.130242 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:35.684655 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:35.684629 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:35.684802 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:35.684637 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:35.684802 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:35.684730 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:35.685447 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:35.684856 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:35.819097 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:35.819069 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-96dhs_5b7c5265-0d16-4645-8524-5c3ba1839641/dns-node-resolver/0.log" Sep 15 07:25:36.609285 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:36.609254 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f9mmh"] Sep 15 07:25:36.609463 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:36.609395 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:36.609548 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:36.609527 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:36.612586 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:36.612525 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m5skf"] Sep 15 07:25:36.612792 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:36.612681 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:36.612864 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:36.612797 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:36.613277 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:36.613255 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v8tt9"] Sep 15 07:25:36.613388 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:36.613333 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:36.613521 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:36.613417 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:37.199069 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:37.199040 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k8pc6_3e78afdb-75af-4944-a28b-7850f145d165/node-ca/0.log" Sep 15 07:25:37.695735 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:37.695661 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:37.695891 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:37.695790 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 15 07:25:37.696366 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:37.696341 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l6k5h" Sep 15 07:25:38.690709 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:38.690684 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:38.691103 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:38.690777 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:38.691216 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:38.691195 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:38.691319 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:38.691285 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:38.691430 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:38.691331 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:38.691430 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:38.691408 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:38.855393 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:38.855346 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22spd" event={"ID":"f4c07647-b791-48b8-bd5e-01de110142d2","Type":"ContainerStarted","Data":"2d6a96b39a49b679e4f3a511ee51156e6b0b05f0594c043f5785d3aab7d80954"} Sep 15 07:25:39.859411 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:39.859377 2577 generic.go:356] "Generic (PLEG): container finished" podID="f4c07647-b791-48b8-bd5e-01de110142d2" containerID="2d6a96b39a49b679e4f3a511ee51156e6b0b05f0594c043f5785d3aab7d80954" exitCode=0 Sep 15 07:25:39.859827 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:39.859450 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22spd" event={"ID":"f4c07647-b791-48b8-bd5e-01de110142d2","Type":"ContainerDied","Data":"2d6a96b39a49b679e4f3a511ee51156e6b0b05f0594c043f5785d3aab7d80954"} Sep 15 07:25:40.685037 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:40.685005 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:40.685037 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:40.685036 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:40.685210 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:40.685013 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:40.685210 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:40.685137 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:40.685294 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:40.685203 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:40.685294 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:40.685274 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:42.275160 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:42.275125 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:42.275632 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.275260 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:42.275632 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.275316 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret podName:e623486b-2e21-4cfc-844d-25fefc0f4418 nodeName:}" failed. No retries permitted until 2026-09-15 07:26:14.275301393 +0000 UTC m=+66.097772775 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret") pod "global-pull-secret-syncer-v8tt9" (UID: "e623486b-2e21-4cfc-844d-25fefc0f4418") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:25:42.376194 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:42.376167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:42.376323 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:42.376209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:42.376323 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.376300 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:42.376323 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.376311 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:25:42.376323 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.376325 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:25:42.376451 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.376333 2577 projected.go:196] Error preparing data for projected volume kube-api-access-g4b7g for pod openshift-network-diagnostics/network-check-target-m5skf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:42.376451 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.376354 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs podName:7ef119f1-8070-4d2a-b511-1fd19295935b nodeName:}" failed. No retries permitted until 2026-09-15 07:26:14.376339369 +0000 UTC m=+66.198810765 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs") pod "network-metrics-daemon-f9mmh" (UID: "7ef119f1-8070-4d2a-b511-1fd19295935b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:25:42.376451 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.376368 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g podName:735140f1-e909-449e-8129-be3752b71965 nodeName:}" failed. No retries permitted until 2026-09-15 07:26:14.376361652 +0000 UTC m=+66.198833034 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4b7g" (UniqueName: "kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g") pod "network-check-target-m5skf" (UID: "735140f1-e909-449e-8129-be3752b71965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:25:42.684661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:42.684631 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:42.684827 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:42.684732 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:42.684827 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.684738 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:42.684827 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:42.684757 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:42.684939 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.684830 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:42.684939 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:42.684882 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:44.684998 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:44.684961 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:44.684998 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:44.684992 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:44.685507 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:44.684969 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:44.685507 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:44.685060 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:44.685507 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:44.685139 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:44.685507 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:44.685205 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:44.873546 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:44.873519 2577 generic.go:356] "Generic (PLEG): container finished" podID="f4c07647-b791-48b8-bd5e-01de110142d2" containerID="7ee841e630732434fba04c994a6c90279ae2b99eb3638c7da70d82097d9727b6" exitCode=0 Sep 15 07:25:44.873688 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:44.873575 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22spd" event={"ID":"f4c07647-b791-48b8-bd5e-01de110142d2","Type":"ContainerDied","Data":"7ee841e630732434fba04c994a6c90279ae2b99eb3638c7da70d82097d9727b6"} Sep 15 07:25:46.684719 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:46.684685 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:46.685261 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:46.684695 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:46.685261 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:46.684815 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:46.685261 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:46.684700 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:46.685261 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:46.684889 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:46.685261 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:46.684952 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:46.879541 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:46.879509 2577 generic.go:356] "Generic (PLEG): container finished" podID="f4c07647-b791-48b8-bd5e-01de110142d2" containerID="e546270aba6d644a9f165776e3f3ea178d2ea0816498b5e5bd5bece82a568a12" exitCode=0 Sep 15 07:25:46.879690 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:46.879576 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22spd" event={"ID":"f4c07647-b791-48b8-bd5e-01de110142d2","Type":"ContainerDied","Data":"e546270aba6d644a9f165776e3f3ea178d2ea0816498b5e5bd5bece82a568a12"} Sep 15 07:25:48.688173 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:48.688138 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:48.688708 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:48.688229 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:48.688708 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:48.688241 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:48.688708 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:48.688236 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:48.688708 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:48.688322 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:48.688708 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:48.688397 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:50.685163 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:50.685130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:50.685637 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:50.685130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:50.685637 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:50.685278 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:50.685637 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:50.685130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:50.685637 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:50.685332 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:50.685637 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:50.685426 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:51.893617 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:51.893585 2577 generic.go:356] "Generic (PLEG): container finished" podID="f4c07647-b791-48b8-bd5e-01de110142d2" containerID="bf9e13d09e2ee0f091306c36b23f33e3426cbf5734cd0715c2b70a8a116282d4" exitCode=0 Sep 15 07:25:51.893617 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:51.893622 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22spd" event={"ID":"f4c07647-b791-48b8-bd5e-01de110142d2","Type":"ContainerDied","Data":"bf9e13d09e2ee0f091306c36b23f33e3426cbf5734cd0715c2b70a8a116282d4"} Sep 15 07:25:52.685212 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:52.685174 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:52.685400 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:52.685222 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:52.685400 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:52.685289 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:52.685400 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:52.685308 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:52.685400 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:52.685361 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:52.685646 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:52.685418 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:52.898887 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:52.898849 2577 generic.go:356] "Generic (PLEG): container finished" podID="f4c07647-b791-48b8-bd5e-01de110142d2" containerID="53777769874df2b67850053e2d2242f8260c297911897f9aec01500fad320f2d" exitCode=0 Sep 15 07:25:52.899228 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:52.898891 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22spd" event={"ID":"f4c07647-b791-48b8-bd5e-01de110142d2","Type":"ContainerDied","Data":"53777769874df2b67850053e2d2242f8260c297911897f9aec01500fad320f2d"} Sep 15 07:25:53.904581 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:53.904546 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22spd" event={"ID":"f4c07647-b791-48b8-bd5e-01de110142d2","Type":"ContainerStarted","Data":"f1f96d37ac18651c0f02a39bc0ce76229f5bd2d84005591ce9871e9b4ed9925c"} Sep 15 07:25:54.684584 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:54.684551 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:54.684742 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:54.684554 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:54.684742 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:54.684678 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:54.684815 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:54.684646 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:54.684815 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:54.684758 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:54.684880 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:54.684851 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:56.684436 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:56.684398 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:56.684833 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:56.684506 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:56.684833 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:56.684515 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:56.684833 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:56.684544 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:56.684833 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:56.684608 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:56.684833 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:56.684664 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:25:58.685607 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:58.685579 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:25:58.686169 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:58.685657 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:25:58.686169 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:58.685667 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:25:58.686169 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:58.685756 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:25:58.686169 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:25:58.685782 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:25:58.686169 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:25:58.685832 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:26:00.684769 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:00.684731 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:26:00.685172 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:00.684732 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:26:00.685172 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:26:00.684838 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v8tt9" podUID="e623486b-2e21-4cfc-844d-25fefc0f4418" Sep 15 07:26:00.685172 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:00.684733 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:26:00.685172 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:26:00.684934 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m5skf" podUID="735140f1-e909-449e-8129-be3752b71965" Sep 15 07:26:00.685172 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:26:00.684995 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9mmh" podUID="7ef119f1-8070-4d2a-b511-1fd19295935b" Sep 15 07:26:01.533681 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.533654 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-38.ec2.internal" event="NodeReady" Sep 15 07:26:01.533834 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.533775 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 15 07:26:01.597391 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.597301 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-22spd" podStartSLOduration=16.27746687 podStartE2EDuration="52.597284531s" totalImagesPullingTime="36.319817661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:53.935086042 +0000 UTC m=+45.757557446" watchObservedRunningTime="2026-09-15 07:26:01.597284531 +0000 UTC m=+53.419755935" Sep 15 07:26:01.597630 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.597614 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6msdh"] Sep 15 07:26:01.599366 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.599351 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6msdh" Sep 15 07:26:01.600919 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.600898 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6xrpp"] Sep 15 07:26:01.602187 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.602166 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 15 07:26:01.602301 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.602187 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-smbg8\"" Sep 15 07:26:01.602350 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.602304 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 15 07:26:01.602641 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.602621 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 15 07:26:01.602732 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.602655 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.605814 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.605788 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 15 07:26:01.605917 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.605885 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 15 07:26:01.606013 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.605993 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 15 07:26:01.606074 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.606060 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wpwdh\"" Sep 15 07:26:01.606530 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.606193 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 15 07:26:01.609895 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.609877 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6msdh"] Sep 15 07:26:01.616103 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.616082 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx6vh\" (UniqueName: \"kubernetes.io/projected/f8933686-b777-4397-bcaa-b0ce3f7f4a9e-kube-api-access-cx6vh\") pod \"ingress-canary-6msdh\" (UID: \"f8933686-b777-4397-bcaa-b0ce3f7f4a9e\") " pod="openshift-ingress-canary/ingress-canary-6msdh" Sep 15 07:26:01.616185 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.616114 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8933686-b777-4397-bcaa-b0ce3f7f4a9e-cert\") pod \"ingress-canary-6msdh\" (UID: \"f8933686-b777-4397-bcaa-b0ce3f7f4a9e\") " pod="openshift-ingress-canary/ingress-canary-6msdh" Sep 15 07:26:01.616185 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.616130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6x87\" (UniqueName: \"kubernetes.io/projected/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-kube-api-access-g6x87\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.616256 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.616194 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-crio-socket\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.616256 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.616225 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-data-volume\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.616314 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.616266 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.616347 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.616318 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.620574 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.620553 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6xrpp"] Sep 15 07:26:01.702749 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.702713 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fhqmn"] Sep 15 07:26:01.704667 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.704651 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.708317 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.708295 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 15 07:26:01.708851 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.708834 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 15 07:26:01.709063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.709048 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8pfvn\"" Sep 15 07:26:01.717599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717580 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.717685 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-tmp-dir\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.717685 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717639 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fjqm\" (UniqueName: \"kubernetes.io/projected/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-kube-api-access-4fjqm\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.717685 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx6vh\" (UniqueName: \"kubernetes.io/projected/f8933686-b777-4397-bcaa-b0ce3f7f4a9e-kube-api-access-cx6vh\") pod \"ingress-canary-6msdh\" (UID: \"f8933686-b777-4397-bcaa-b0ce3f7f4a9e\") " pod="openshift-ingress-canary/ingress-canary-6msdh" Sep 15 07:26:01.717814 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717793 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8933686-b777-4397-bcaa-b0ce3f7f4a9e-cert\") pod \"ingress-canary-6msdh\" (UID: \"f8933686-b777-4397-bcaa-b0ce3f7f4a9e\") " pod="openshift-ingress-canary/ingress-canary-6msdh" Sep 15 07:26:01.717850 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717830 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6x87\" (UniqueName: \"kubernetes.io/projected/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-kube-api-access-g6x87\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.717899 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717885 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-config-volume\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.717934 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717922 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-crio-socket\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.717982 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717949 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-data-volume\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.717982 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717972 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.718069 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.717991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-metrics-tls\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.718155 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.718133 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-crio-socket\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.718200 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.718165 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.718253 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.718239 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-data-volume\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.721564 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.721541 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.721564 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.721551 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8933686-b777-4397-bcaa-b0ce3f7f4a9e-cert\") pod \"ingress-canary-6msdh\" (UID: \"f8933686-b777-4397-bcaa-b0ce3f7f4a9e\") " pod="openshift-ingress-canary/ingress-canary-6msdh" Sep 15 07:26:01.731336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.731312 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fhqmn"] Sep 15 07:26:01.736930 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.736907 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6x87\" (UniqueName: \"kubernetes.io/projected/5b36d217-aaff-4fa0-bfa6-11a85b51b1bc-kube-api-access-g6x87\") pod \"insights-runtime-extractor-6xrpp\" (UID: \"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc\") " pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:01.738123 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.738102 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx6vh\" (UniqueName: \"kubernetes.io/projected/f8933686-b777-4397-bcaa-b0ce3f7f4a9e-kube-api-access-cx6vh\") pod \"ingress-canary-6msdh\" (UID: \"f8933686-b777-4397-bcaa-b0ce3f7f4a9e\") " pod="openshift-ingress-canary/ingress-canary-6msdh" Sep 15 07:26:01.819268 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.819228 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fjqm\" (UniqueName: \"kubernetes.io/projected/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-kube-api-access-4fjqm\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.819428 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.819285 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-config-volume\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.819428 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.819315 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-metrics-tls\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.819428 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.819342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-tmp-dir\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.819743 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.819721 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-tmp-dir\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.819930 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.819914 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-config-volume\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.821617 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.821600 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-metrics-tls\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.832400 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.832376 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fjqm\" (UniqueName: \"kubernetes.io/projected/0916e3c6-af2a-4b29-8f5f-b4eca5a04b77-kube-api-access-4fjqm\") pod \"dns-default-fhqmn\" (UID: \"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77\") " pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:01.909380 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.909332 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6msdh" Sep 15 07:26:01.914828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:01.914813 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6xrpp" Sep 15 07:26:02.010117 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.010077 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq"] Sep 15 07:26:02.012034 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.012015 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" Sep 15 07:26:02.012906 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.012875 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:02.018217 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.018202 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 15 07:26:02.018384 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.018365 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xscbh\"" Sep 15 07:26:02.020151 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.020133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/30877835-7945-4800-81cb-19ef018d2d72-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-srvxq\" (UID: \"30877835-7945-4800-81cb-19ef018d2d72\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" Sep 15 07:26:02.024519 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.024498 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq"] Sep 15 07:26:02.121165 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.121132 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/30877835-7945-4800-81cb-19ef018d2d72-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-srvxq\" (UID: \"30877835-7945-4800-81cb-19ef018d2d72\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" Sep 15 07:26:02.121274 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:26:02.121235 2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 15 07:26:02.121314 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:26:02.121290 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30877835-7945-4800-81cb-19ef018d2d72-tls-certificates podName:30877835-7945-4800-81cb-19ef018d2d72 nodeName:}" failed. No retries permitted until 2026-09-15 07:26:02.621274812 +0000 UTC m=+54.443746194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/30877835-7945-4800-81cb-19ef018d2d72-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-srvxq" (UID: "30877835-7945-4800-81cb-19ef018d2d72") : secret "prometheus-operator-admission-webhook-tls" not found Sep 15 07:26:02.194162 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.194135 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6msdh"] Sep 15 07:26:02.196602 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.196579 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6xrpp"] Sep 15 07:26:02.199321 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:02.199293 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8933686_b777_4397_bcaa_b0ce3f7f4a9e.slice/crio-dafbb53e92263d477cf17557029e29109a745a541cdb979e497a0075387daaca WatchSource:0}: Error finding container dafbb53e92263d477cf17557029e29109a745a541cdb979e497a0075387daaca: Status 404 returned error can't find the container with id dafbb53e92263d477cf17557029e29109a745a541cdb979e497a0075387daaca Sep 15 07:26:02.201182 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:02.201162 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b36d217_aaff_4fa0_bfa6_11a85b51b1bc.slice/crio-b9e1af556e3bca052ab651b0b1cce93022aaac983f490009736050272acde79e WatchSource:0}: Error finding container b9e1af556e3bca052ab651b0b1cce93022aaac983f490009736050272acde79e: Status 404 returned error can't find the container with id b9e1af556e3bca052ab651b0b1cce93022aaac983f490009736050272acde79e Sep 15 07:26:02.282993 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.282909 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fhqmn"] Sep 15 07:26:02.285179 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:02.285148 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0916e3c6_af2a_4b29_8f5f_b4eca5a04b77.slice/crio-d70d72c1531ba634c8c6fc1042c2c85d70ea3acddb79b1060485542f56914c13 WatchSource:0}: Error finding container d70d72c1531ba634c8c6fc1042c2c85d70ea3acddb79b1060485542f56914c13: Status 404 returned error can't find the container with id d70d72c1531ba634c8c6fc1042c2c85d70ea3acddb79b1060485542f56914c13 Sep 15 07:26:02.625328 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.625241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/30877835-7945-4800-81cb-19ef018d2d72-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-srvxq\" (UID: \"30877835-7945-4800-81cb-19ef018d2d72\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" Sep 15 07:26:02.627536 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.627516 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/30877835-7945-4800-81cb-19ef018d2d72-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-srvxq\" (UID: \"30877835-7945-4800-81cb-19ef018d2d72\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" Sep 15 07:26:02.684609 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.684580 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:26:02.684753 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.684580 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:26:02.685046 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.684580 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:26:02.689302 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.689278 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 07:26:02.690279 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.690260 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4f6cz\"" Sep 15 07:26:02.690385 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.690262 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 07:26:02.690891 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.690871 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 07:26:02.690987 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.690912 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 07:26:02.691063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.690914 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vrc7c\"" Sep 15 07:26:02.920915 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.920831 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" Sep 15 07:26:02.925535 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.925507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fhqmn" event={"ID":"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77","Type":"ContainerStarted","Data":"d70d72c1531ba634c8c6fc1042c2c85d70ea3acddb79b1060485542f56914c13"} Sep 15 07:26:02.927745 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.927696 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xrpp" event={"ID":"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc","Type":"ContainerStarted","Data":"24f6f048e33d7204351068d1d967fdfb0ad20221fec79acbe66901fbb30fe3bd"} Sep 15 07:26:02.927858 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.927776 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xrpp" event={"ID":"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc","Type":"ContainerStarted","Data":"b9e1af556e3bca052ab651b0b1cce93022aaac983f490009736050272acde79e"} Sep 15 07:26:02.929086 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:02.929062 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6msdh" event={"ID":"f8933686-b777-4397-bcaa-b0ce3f7f4a9e","Type":"ContainerStarted","Data":"dafbb53e92263d477cf17557029e29109a745a541cdb979e497a0075387daaca"} Sep 15 07:26:03.200686 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:03.200651 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq"] Sep 15 07:26:03.203602 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:03.203574 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30877835_7945_4800_81cb_19ef018d2d72.slice/crio-9def270b9cfb60675f2787840fef78becbb2d83397ba556fd80a6c73bb9b9db0 WatchSource:0}: Error finding container 9def270b9cfb60675f2787840fef78becbb2d83397ba556fd80a6c73bb9b9db0: Status 404 returned error can't find the container with id 9def270b9cfb60675f2787840fef78becbb2d83397ba556fd80a6c73bb9b9db0 Sep 15 07:26:03.934684 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:03.934643 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xrpp" event={"ID":"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc","Type":"ContainerStarted","Data":"39004a4e9d8e45c2da38f9d9a3487bdf81500037dd65df779e2e12d1c4662833"} Sep 15 07:26:03.936109 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:03.936037 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" event={"ID":"30877835-7945-4800-81cb-19ef018d2d72","Type":"ContainerStarted","Data":"9def270b9cfb60675f2787840fef78becbb2d83397ba556fd80a6c73bb9b9db0"} Sep 15 07:26:05.944049 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:05.944008 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6msdh" event={"ID":"f8933686-b777-4397-bcaa-b0ce3f7f4a9e","Type":"ContainerStarted","Data":"3e137cb4e6d61a74e07830bf0b4bfb7f988b9bee9fef922d8d969d16cba8b2b1"} Sep 15 07:26:05.951355 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:05.951326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xrpp" event={"ID":"5b36d217-aaff-4fa0-bfa6-11a85b51b1bc","Type":"ContainerStarted","Data":"f8f8035be877a909f3f0603698a9dc103bc4c56c554d2e962b5ae6d47a05dcbe"} Sep 15 07:26:05.952890 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:05.952862 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" event={"ID":"30877835-7945-4800-81cb-19ef018d2d72","Type":"ContainerStarted","Data":"79ea81366052c609d72af92bb369c0f5d099a1e3129011bb06791d4148135ca4"} Sep 15 07:26:05.953120 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:05.953102 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" Sep 15 07:26:05.958851 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:05.958823 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" Sep 15 07:26:05.967411 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:05.967371 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6msdh" podStartSLOduration=2.087328408 podStartE2EDuration="4.96735959s" totalImagesPullingTime="2.880031182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:05.967025731 +0000 UTC m=+57.789497141" watchObservedRunningTime="2026-09-15 07:26:05.96735959 +0000 UTC m=+57.789830995" Sep 15 07:26:05.995430 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:05.995386 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6xrpp" podStartSLOduration=2.247740236 podStartE2EDuration="4.99537264s" totalImagesPullingTime="2.747632404s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:26:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:05.995028861 +0000 UTC m=+57.817500268" watchObservedRunningTime="2026-09-15 07:26:05.99537264 +0000 UTC m=+57.817844045" Sep 15 07:26:06.022372 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:06.022319 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-srvxq" podStartSLOduration=3.144978958 podStartE2EDuration="5.022299544s" totalImagesPullingTime="1.877320586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:06.021832093 +0000 UTC m=+57.844303498" watchObservedRunningTime="2026-09-15 07:26:06.022299544 +0000 UTC m=+57.844770952" Sep 15 07:26:06.861378 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:06.861347 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4k5xc" Sep 15 07:26:07.122226 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.122150 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w"] Sep 15 07:26:07.124978 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.124955 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.130570 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.130384 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 15 07:26:07.130570 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.130427 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 15 07:26:07.131312 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.130915 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 15 07:26:07.131312 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.131138 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nwxqr\"" Sep 15 07:26:07.131312 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.131299 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 15 07:26:07.141998 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.141983 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 15 07:26:07.148974 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.148946 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w"] Sep 15 07:26:07.160280 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.160257 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/875e3815-9b50-4a3b-bb1b-b58ead23ea74-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.160345 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.160291 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/875e3815-9b50-4a3b-bb1b-b58ead23ea74-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.160345 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.160327 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/875e3815-9b50-4a3b-bb1b-b58ead23ea74-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.160427 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.160404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t486h\" (UniqueName: \"kubernetes.io/projected/875e3815-9b50-4a3b-bb1b-b58ead23ea74-kube-api-access-t486h\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.261684 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.261653 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/875e3815-9b50-4a3b-bb1b-b58ead23ea74-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.261684 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.261686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/875e3815-9b50-4a3b-bb1b-b58ead23ea74-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.261913 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.261720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/875e3815-9b50-4a3b-bb1b-b58ead23ea74-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.261913 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.261780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t486h\" (UniqueName: \"kubernetes.io/projected/875e3815-9b50-4a3b-bb1b-b58ead23ea74-kube-api-access-t486h\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.262413 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.262388 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/875e3815-9b50-4a3b-bb1b-b58ead23ea74-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.264035 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.264008 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/875e3815-9b50-4a3b-bb1b-b58ead23ea74-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.264116 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.264093 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/875e3815-9b50-4a3b-bb1b-b58ead23ea74-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.271759 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.271739 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t486h\" (UniqueName: \"kubernetes.io/projected/875e3815-9b50-4a3b-bb1b-b58ead23ea74-kube-api-access-t486h\") pod \"prometheus-operator-68bcd7f987-qdb5w\" (UID: \"875e3815-9b50-4a3b-bb1b-b58ead23ea74\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.436915 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.436884 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" Sep 15 07:26:07.709864 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.709840 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w"] Sep 15 07:26:07.710921 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:07.710898 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875e3815_9b50_4a3b_bb1b_b58ead23ea74.slice/crio-53aabdc7e127c951872ee9569236db63bca270c2ed9ffddf0069b2bdd419a7de WatchSource:0}: Error finding container 53aabdc7e127c951872ee9569236db63bca270c2ed9ffddf0069b2bdd419a7de: Status 404 returned error can't find the container with id 53aabdc7e127c951872ee9569236db63bca270c2ed9ffddf0069b2bdd419a7de Sep 15 07:26:07.960355 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.960266 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" event={"ID":"875e3815-9b50-4a3b-bb1b-b58ead23ea74","Type":"ContainerStarted","Data":"53aabdc7e127c951872ee9569236db63bca270c2ed9ffddf0069b2bdd419a7de"} Sep 15 07:26:07.961924 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.961891 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fhqmn" event={"ID":"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77","Type":"ContainerStarted","Data":"bc9a801fb9ed0b3a0937ae29f21135d4eb3c3bec50d27f779847e3812f234a27"} Sep 15 07:26:07.962014 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.961944 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fhqmn" event={"ID":"0916e3c6-af2a-4b29-8f5f-b4eca5a04b77","Type":"ContainerStarted","Data":"03418f83e13c5b8a6dc6bc1ba3e2f852bac0a34d984c10017f78110532eb56d1"} Sep 15 07:26:07.993897 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:07.993854 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fhqmn" podStartSLOduration=2.295320063 podStartE2EDuration="6.993840288s" totalImagesPullingTime="4.698520225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:07.992693345 +0000 UTC m=+59.815164749" watchObservedRunningTime="2026-09-15 07:26:07.993840288 +0000 UTC m=+59.816311692" Sep 15 07:26:08.964341 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:08.964310 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:09.704226 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:09.704192 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-vzq75"] Sep 15 07:26:09.706739 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:09.706723 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-vzq75" Sep 15 07:26:09.709905 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:09.709888 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 15 07:26:09.710002 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:09.709948 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 15 07:26:09.710072 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:09.710055 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-vgf9f\"" Sep 15 07:26:09.718533 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:09.718511 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-vzq75"] Sep 15 07:26:09.781475 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:09.781443 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rntbx\" (UniqueName: \"kubernetes.io/projected/9bf86a8e-64a0-4edf-be9b-1543f8ab172a-kube-api-access-rntbx\") pod \"downloads-76f99584f4-vzq75\" (UID: \"9bf86a8e-64a0-4edf-be9b-1543f8ab172a\") " pod="openshift-console/downloads-76f99584f4-vzq75" Sep 15 07:26:09.882599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:09.882571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rntbx\" (UniqueName: \"kubernetes.io/projected/9bf86a8e-64a0-4edf-be9b-1543f8ab172a-kube-api-access-rntbx\") pod \"downloads-76f99584f4-vzq75\" (UID: \"9bf86a8e-64a0-4edf-be9b-1543f8ab172a\") " pod="openshift-console/downloads-76f99584f4-vzq75" Sep 15 07:26:09.892060 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:09.892028 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rntbx\" (UniqueName: \"kubernetes.io/projected/9bf86a8e-64a0-4edf-be9b-1543f8ab172a-kube-api-access-rntbx\") pod \"downloads-76f99584f4-vzq75\" (UID: \"9bf86a8e-64a0-4edf-be9b-1543f8ab172a\") " pod="openshift-console/downloads-76f99584f4-vzq75" Sep 15 07:26:10.015834 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:10.015762 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-vzq75" Sep 15 07:26:10.300818 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:10.300748 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-vzq75"] Sep 15 07:26:10.304055 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:10.304019 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf86a8e_64a0_4edf_be9b_1543f8ab172a.slice/crio-955e58e39683ad27f30a02c70b50823068e3c93303033cc922e83dc77fd658ba WatchSource:0}: Error finding container 955e58e39683ad27f30a02c70b50823068e3c93303033cc922e83dc77fd658ba: Status 404 returned error can't find the container with id 955e58e39683ad27f30a02c70b50823068e3c93303033cc922e83dc77fd658ba Sep 15 07:26:10.970948 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:10.970915 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-vzq75" event={"ID":"9bf86a8e-64a0-4edf-be9b-1543f8ab172a","Type":"ContainerStarted","Data":"955e58e39683ad27f30a02c70b50823068e3c93303033cc922e83dc77fd658ba"} Sep 15 07:26:12.978762 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:12.978722 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" event={"ID":"875e3815-9b50-4a3b-bb1b-b58ead23ea74","Type":"ContainerStarted","Data":"62a0eaf88fb1b1ab5f87a2c690f43d9cfb71f9fed0ee700cf29f9e7c6673d32b"} Sep 15 07:26:12.979131 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:12.978787 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" event={"ID":"875e3815-9b50-4a3b-bb1b-b58ead23ea74","Type":"ContainerStarted","Data":"cc3e21b62eee11d3a67d08ada0bf7800c76d225a3506287ca584b78a9acc3306"} Sep 15 07:26:13.007400 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:13.007355 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-qdb5w" podStartSLOduration=1.3788721449999999 podStartE2EDuration="6.007340087s" totalImagesPullingTime="4.628467942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:13.006196881 +0000 UTC m=+64.828668286" watchObservedRunningTime="2026-09-15 07:26:13.007340087 +0000 UTC m=+64.829811492" Sep 15 07:26:14.316104 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.316064 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:26:14.319219 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.319199 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 07:26:14.329249 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.329223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e623486b-2e21-4cfc-844d-25fefc0f4418-original-pull-secret\") pod \"global-pull-secret-syncer-v8tt9\" (UID: \"e623486b-2e21-4cfc-844d-25fefc0f4418\") " pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:26:14.396920 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.396897 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v8tt9" Sep 15 07:26:14.416610 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.416584 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:26:14.416707 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.416625 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:26:14.422723 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.422690 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-657ccdbdf6-22hqp"] Sep 15 07:26:14.422834 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.422749 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 07:26:14.423776 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.423759 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 07:26:14.425179 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.425161 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.429343 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.429165 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ef119f1-8070-4d2a-b511-1fd19295935b-metrics-certs\") pod \"network-metrics-daemon-f9mmh\" (UID: \"7ef119f1-8070-4d2a-b511-1fd19295935b\") " pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:26:14.429494 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.429454 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 15 07:26:14.430510 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.430174 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 15 07:26:14.430510 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.430283 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 15 07:26:14.430510 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.430426 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 15 07:26:14.433408 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.431075 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 07:26:14.433408 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.431230 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 15 07:26:14.433408 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.431968 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qxjsw\"" Sep 15 07:26:14.440606 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.440588 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-657ccdbdf6-22hqp"] Sep 15 07:26:14.440991 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.440971 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4b7g\" (UniqueName: \"kubernetes.io/projected/735140f1-e909-449e-8129-be3752b71965-kube-api-access-g4b7g\") pod \"network-check-target-m5skf\" (UID: \"735140f1-e909-449e-8129-be3752b71965\") " pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:26:14.517169 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.517134 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-serving-cert\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.517169 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.517168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-service-ca\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.517341 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.517188 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-oauth-serving-cert\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.517341 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.517209 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-console-config\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.517341 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.517250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vhbc\" (UniqueName: \"kubernetes.io/projected/d18b7df9-0525-4691-afc3-e420e069a100-kube-api-access-7vhbc\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.517341 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.517272 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-oauth-config\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.617747 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.617676 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vhbc\" (UniqueName: \"kubernetes.io/projected/d18b7df9-0525-4691-afc3-e420e069a100-kube-api-access-7vhbc\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.617747 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.617714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-oauth-config\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.617747 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.617737 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-serving-cert\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.617956 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.617753 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-service-ca\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.617956 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.617775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-oauth-serving-cert\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.617956 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.617805 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-console-config\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.618529 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.618467 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-oauth-serving-cert\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.618642 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.618621 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-service-ca\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.618709 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.618621 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-console-config\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.620132 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.620111 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-oauth-config\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.620223 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.620187 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-serving-cert\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.645353 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.645328 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vhbc\" (UniqueName: \"kubernetes.io/projected/d18b7df9-0525-4691-afc3-e420e069a100-kube-api-access-7vhbc\") pod \"console-657ccdbdf6-22hqp\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.703124 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.703104 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v8tt9"] Sep 15 07:26:14.706442 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:14.706419 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode623486b_2e21_4cfc_844d_25fefc0f4418.slice/crio-4b5a5f0b1d25fa424cad51111dfb8492f36b239eb5b887cc03dbf139a770391d WatchSource:0}: Error finding container 4b5a5f0b1d25fa424cad51111dfb8492f36b239eb5b887cc03dbf139a770391d: Status 404 returned error can't find the container with id 4b5a5f0b1d25fa424cad51111dfb8492f36b239eb5b887cc03dbf139a770391d Sep 15 07:26:14.707001 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.706982 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4f6cz\"" Sep 15 07:26:14.713224 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.713203 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9mmh" Sep 15 07:26:14.713909 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.713892 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vrc7c\"" Sep 15 07:26:14.720543 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.720525 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:26:14.749019 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.748989 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:14.750690 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.750669 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr"] Sep 15 07:26:14.753786 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.753336 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.758242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.757755 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zpkx5\"" Sep 15 07:26:14.758242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.758062 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 15 07:26:14.758242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.758170 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 15 07:26:14.780670 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.780571 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr"] Sep 15 07:26:14.812369 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.812342 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5"] Sep 15 07:26:14.815432 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.815413 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:14.820005 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.819824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnjsh\" (UniqueName: \"kubernetes.io/projected/660ea539-a67b-4401-8041-b5cc09913ce0-kube-api-access-jnjsh\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.820005 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.819877 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/660ea539-a67b-4401-8041-b5cc09913ce0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.820005 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.819927 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/660ea539-a67b-4401-8041-b5cc09913ce0-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.820005 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.819958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/660ea539-a67b-4401-8041-b5cc09913ce0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.826086 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.826064 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 15 07:26:14.826267 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.826249 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 15 07:26:14.828622 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.827343 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bdsl8\"" Sep 15 07:26:14.828622 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.827504 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 15 07:26:14.866200 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.866176 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5"] Sep 15 07:26:14.886048 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.885989 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6z2wq"] Sep 15 07:26:14.887986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.887972 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.895163 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.895135 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 15 07:26:14.895279 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.895213 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b9qbs\"" Sep 15 07:26:14.896253 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.896239 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 15 07:26:14.896301 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.896251 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 15 07:26:14.920907 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.920879 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6458032e-6e59-405f-908a-a59993c13da8-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:14.921017 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.920916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-tls\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.921017 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.920943 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27l6j\" (UniqueName: \"kubernetes.io/projected/6458032e-6e59-405f-908a-a59993c13da8-kube-api-access-27l6j\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:14.921017 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921002 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnjsh\" (UniqueName: \"kubernetes.io/projected/660ea539-a67b-4401-8041-b5cc09913ce0-kube-api-access-jnjsh\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.921156 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921084 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6458032e-6e59-405f-908a-a59993c13da8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:14.921156 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/660ea539-a67b-4401-8041-b5cc09913ce0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.921156 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-accelerators-collector-config\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.921156 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.921325 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921185 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/660ea539-a67b-4401-8041-b5cc09913ce0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.921325 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/660ea539-a67b-4401-8041-b5cc09913ce0-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.921325 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921225 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6458032e-6e59-405f-908a-a59993c13da8-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:14.921325 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921258 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-textfile\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.921325 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921299 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6458032e-6e59-405f-908a-a59993c13da8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:14.921579 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921331 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp4ln\" (UniqueName: \"kubernetes.io/projected/567be93d-745d-4dda-a1e2-44c77d30da63-kube-api-access-dp4ln\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.921579 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921360 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/567be93d-745d-4dda-a1e2-44c77d30da63-root\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.921579 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/567be93d-745d-4dda-a1e2-44c77d30da63-sys\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.921579 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921410 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6458032e-6e59-405f-908a-a59993c13da8-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:14.921579 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-wtmp\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.921579 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921472 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/567be93d-745d-4dda-a1e2-44c77d30da63-metrics-client-ca\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:14.921830 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.921814 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/660ea539-a67b-4401-8041-b5cc09913ce0-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.923280 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.923262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/660ea539-a67b-4401-8041-b5cc09913ce0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.923470 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.923454 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/660ea539-a67b-4401-8041-b5cc09913ce0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.940331 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.940311 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnjsh\" (UniqueName: \"kubernetes.io/projected/660ea539-a67b-4401-8041-b5cc09913ce0-kube-api-access-jnjsh\") pod \"openshift-state-metrics-67dcf8c979-vmnlr\" (UID: \"660ea539-a67b-4401-8041-b5cc09913ce0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:14.986209 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:14.986184 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v8tt9" event={"ID":"e623486b-2e21-4cfc-844d-25fefc0f4418","Type":"ContainerStarted","Data":"4b5a5f0b1d25fa424cad51111dfb8492f36b239eb5b887cc03dbf139a770391d"} Sep 15 07:26:15.011316 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.011291 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m5skf"] Sep 15 07:26:15.016103 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:15.016080 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735140f1_e909_449e_8129_be3752b71965.slice/crio-a7dc2b7acde7b3fee0659d73a4db327f206c614cf2e7dd304fb64b95f3cbd42c WatchSource:0}: Error finding container a7dc2b7acde7b3fee0659d73a4db327f206c614cf2e7dd304fb64b95f3cbd42c: Status 404 returned error can't find the container with id a7dc2b7acde7b3fee0659d73a4db327f206c614cf2e7dd304fb64b95f3cbd42c Sep 15 07:26:15.022652 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022632 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6458032e-6e59-405f-908a-a59993c13da8-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.022713 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022667 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-textfile\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.022713 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022695 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6458032e-6e59-405f-908a-a59993c13da8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.022821 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022803 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp4ln\" (UniqueName: \"kubernetes.io/projected/567be93d-745d-4dda-a1e2-44c77d30da63-kube-api-access-dp4ln\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.022881 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/567be93d-745d-4dda-a1e2-44c77d30da63-root\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.022936 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/567be93d-745d-4dda-a1e2-44c77d30da63-sys\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.022936 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022903 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6458032e-6e59-405f-908a-a59993c13da8-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.022936 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022925 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-textfile\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023088 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022929 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-wtmp\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023088 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022993 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/567be93d-745d-4dda-a1e2-44c77d30da63-sys\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023088 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.022997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/567be93d-745d-4dda-a1e2-44c77d30da63-metrics-client-ca\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023088 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023078 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6458032e-6e59-405f-908a-a59993c13da8-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.023290 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023102 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-wtmp\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023290 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-tls\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023290 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023137 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27l6j\" (UniqueName: \"kubernetes.io/projected/6458032e-6e59-405f-908a-a59993c13da8-kube-api-access-27l6j\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.023290 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023145 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/567be93d-745d-4dda-a1e2-44c77d30da63-root\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023290 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6458032e-6e59-405f-908a-a59993c13da8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.023290 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-accelerators-collector-config\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023290 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023693 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023381 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6458032e-6e59-405f-908a-a59993c13da8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.023693 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023417 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6458032e-6e59-405f-908a-a59993c13da8-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.023693 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023574 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/567be93d-745d-4dda-a1e2-44c77d30da63-metrics-client-ca\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.023693 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023668 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6458032e-6e59-405f-908a-a59993c13da8-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.023997 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.023968 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-accelerators-collector-config\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.025525 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.025498 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-tls\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.025622 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.025548 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6458032e-6e59-405f-908a-a59993c13da8-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.025686 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.025670 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6458032e-6e59-405f-908a-a59993c13da8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.025727 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.025669 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/567be93d-745d-4dda-a1e2-44c77d30da63-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.033651 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.033622 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f9mmh"] Sep 15 07:26:15.036023 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:15.036003 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef119f1_8070_4d2a_b511_1fd19295935b.slice/crio-b94ecb8f0d2a7e776011df027b30f019202cfff41e8b8737e32c8aa56380fba8 WatchSource:0}: Error finding container b94ecb8f0d2a7e776011df027b30f019202cfff41e8b8737e32c8aa56380fba8: Status 404 returned error can't find the container with id b94ecb8f0d2a7e776011df027b30f019202cfff41e8b8737e32c8aa56380fba8 Sep 15 07:26:15.050180 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.050160 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp4ln\" (UniqueName: \"kubernetes.io/projected/567be93d-745d-4dda-a1e2-44c77d30da63-kube-api-access-dp4ln\") pod \"node-exporter-6z2wq\" (UID: \"567be93d-745d-4dda-a1e2-44c77d30da63\") " pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.051637 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.051620 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27l6j\" (UniqueName: \"kubernetes.io/projected/6458032e-6e59-405f-908a-a59993c13da8-kube-api-access-27l6j\") pod \"kube-state-metrics-59fbdd77d-f9qt5\" (UID: \"6458032e-6e59-405f-908a-a59993c13da8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.054259 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.054242 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-657ccdbdf6-22hqp"] Sep 15 07:26:15.056225 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:15.056203 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18b7df9_0525_4691_afc3_e420e069a100.slice/crio-306a5771beccb6b92a91a0185269f832af83374c090b4efc3eacee1b0910f66b WatchSource:0}: Error finding container 306a5771beccb6b92a91a0185269f832af83374c090b4efc3eacee1b0910f66b: Status 404 returned error can't find the container with id 306a5771beccb6b92a91a0185269f832af83374c090b4efc3eacee1b0910f66b Sep 15 07:26:15.064433 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.064417 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" Sep 15 07:26:15.126339 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.126134 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" Sep 15 07:26:15.196681 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.196599 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6z2wq" Sep 15 07:26:15.207437 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:15.207410 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod567be93d_745d_4dda_a1e2_44c77d30da63.slice/crio-f074cfeb017c85e21d192a94a42a491c43d80a53bc3a0bc1619f97f31fc0a3e1 WatchSource:0}: Error finding container f074cfeb017c85e21d192a94a42a491c43d80a53bc3a0bc1619f97f31fc0a3e1: Status 404 returned error can't find the container with id f074cfeb017c85e21d192a94a42a491c43d80a53bc3a0bc1619f97f31fc0a3e1 Sep 15 07:26:15.335732 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.335703 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr"] Sep 15 07:26:15.338563 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:15.338527 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod660ea539_a67b_4401_8041_b5cc09913ce0.slice/crio-7466f1f4c97f6e94c276b8250af8acee32d685c9591d104de74df975dba609e9 WatchSource:0}: Error finding container 7466f1f4c97f6e94c276b8250af8acee32d685c9591d104de74df975dba609e9: Status 404 returned error can't find the container with id 7466f1f4c97f6e94c276b8250af8acee32d685c9591d104de74df975dba609e9 Sep 15 07:26:15.416825 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.416795 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5"] Sep 15 07:26:15.420092 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:15.420056 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6458032e_6e59_405f_908a_a59993c13da8.slice/crio-e9a8df17bb0120e9fdaed20f86c1987b856e5ec1ef7db5c2164471c80c0b2113 WatchSource:0}: Error finding container e9a8df17bb0120e9fdaed20f86c1987b856e5ec1ef7db5c2164471c80c0b2113: Status 404 returned error can't find the container with id e9a8df17bb0120e9fdaed20f86c1987b856e5ec1ef7db5c2164471c80c0b2113 Sep 15 07:26:15.646082 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.646036 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:26:15.656796 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.656764 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.660925 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.660896 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 15 07:26:15.661153 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.661132 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 15 07:26:15.661362 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.661345 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 15 07:26:15.661575 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.661556 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 15 07:26:15.661759 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.661739 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 15 07:26:15.661840 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.661554 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j46r7\"" Sep 15 07:26:15.661840 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.661672 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 15 07:26:15.661916 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.661696 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 15 07:26:15.661916 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.661719 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 15 07:26:15.661974 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.661615 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 15 07:26:15.677347 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.677324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:26:15.728758 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.728719 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-web-config\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.728955 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.728764 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.728955 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.728794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.728955 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.728825 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.728955 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.728857 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.728955 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.728888 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-tls-assets\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.729364 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.728960 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.729364 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.729004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc7ft\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-kube-api-access-lc7ft\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.729364 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.729031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-volume\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.729364 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.729058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.729364 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.729107 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-out\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.729364 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.729151 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.729364 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.729177 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.831543 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831504 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.831737 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831564 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.831737 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-tls-assets\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.831737 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831642 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.831737 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc7ft\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-kube-api-access-lc7ft\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.831737 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831700 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-volume\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.831737 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831727 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.832063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-out\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.832063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.832063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831823 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.832063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831879 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-web-config\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.832063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831902 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.832063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.831931 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.832921 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.832859 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.833124 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.833072 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.833124 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.833090 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.833124 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:26:15.833103 2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 15 07:26:15.833336 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:26:15.833173 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-main-tls podName:99cbd8ed-514f-4240-9bd2-f9ccd6c88930 nodeName:}" failed. No retries permitted until 2026-09-15 07:26:16.333151261 +0000 UTC m=+68.155622658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930") : secret "alertmanager-main-tls" not found Sep 15 07:26:15.835817 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.835772 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.836900 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.836875 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-tls-assets\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.837158 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.837113 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-out\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.837509 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.837445 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.838008 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.837969 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.838326 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.838148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-volume\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.839216 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.839182 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.839961 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.839941 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-web-config\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.859009 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.858981 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc7ft\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-kube-api-access-lc7ft\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:15.993863 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.993669 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" event={"ID":"660ea539-a67b-4401-8041-b5cc09913ce0","Type":"ContainerStarted","Data":"8d0f4be1d51af5c8a31bf59318dd57fd612f31f0421d847c5256b676a001944e"} Sep 15 07:26:15.993863 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.993745 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" event={"ID":"660ea539-a67b-4401-8041-b5cc09913ce0","Type":"ContainerStarted","Data":"aec6674adadffc22e9d876a9a8fe489eb13c9d6a9117fd3def0d9607d5683632"} Sep 15 07:26:15.993863 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.993784 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" event={"ID":"660ea539-a67b-4401-8041-b5cc09913ce0","Type":"ContainerStarted","Data":"7466f1f4c97f6e94c276b8250af8acee32d685c9591d104de74df975dba609e9"} Sep 15 07:26:15.996010 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.995836 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" event={"ID":"6458032e-6e59-405f-908a-a59993c13da8","Type":"ContainerStarted","Data":"e9a8df17bb0120e9fdaed20f86c1987b856e5ec1ef7db5c2164471c80c0b2113"} Sep 15 07:26:15.998094 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:15.998000 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6z2wq" event={"ID":"567be93d-745d-4dda-a1e2-44c77d30da63","Type":"ContainerStarted","Data":"f074cfeb017c85e21d192a94a42a491c43d80a53bc3a0bc1619f97f31fc0a3e1"} Sep 15 07:26:16.000408 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.000348 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657ccdbdf6-22hqp" event={"ID":"d18b7df9-0525-4691-afc3-e420e069a100","Type":"ContainerStarted","Data":"306a5771beccb6b92a91a0185269f832af83374c090b4efc3eacee1b0910f66b"} Sep 15 07:26:16.002201 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.002174 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f9mmh" event={"ID":"7ef119f1-8070-4d2a-b511-1fd19295935b","Type":"ContainerStarted","Data":"b94ecb8f0d2a7e776011df027b30f019202cfff41e8b8737e32c8aa56380fba8"} Sep 15 07:26:16.004048 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.003981 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m5skf" event={"ID":"735140f1-e909-449e-8129-be3752b71965","Type":"ContainerStarted","Data":"a7dc2b7acde7b3fee0659d73a4db327f206c614cf2e7dd304fb64b95f3cbd42c"} Sep 15 07:26:16.336676 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.336501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:16.339782 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.339752 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:16.570216 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.569998 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:26:16.678334 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.678254 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-54c44dc67-kpmp5"] Sep 15 07:26:16.681388 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.681363 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.691207 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.689892 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 15 07:26:16.691207 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.690117 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 15 07:26:16.691207 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.690293 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 15 07:26:16.691207 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.690460 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 15 07:26:16.691499 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.691277 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bmjzp\"" Sep 15 07:26:16.691669 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.691504 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6sdrubet0d4a2\"" Sep 15 07:26:16.691747 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.691717 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 15 07:26:16.713931 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.713903 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54c44dc67-kpmp5"] Sep 15 07:26:16.739690 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.739638 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.739816 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.739692 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/098e566e-0737-4066-bb61-44a819786dca-metrics-client-ca\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.739816 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.739753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.739927 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.739856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.739927 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.739898 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.740125 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.739971 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-tls\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.740125 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.740050 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7jdg\" (UniqueName: \"kubernetes.io/projected/098e566e-0737-4066-bb61-44a819786dca-kube-api-access-g7jdg\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.740125 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.740094 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-grpc-tls\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.840701 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.840667 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/098e566e-0737-4066-bb61-44a819786dca-metrics-client-ca\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.840883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.840758 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.840883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.840808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.840883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.840836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.840883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.840879 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-tls\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.841092 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.840920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7jdg\" (UniqueName: \"kubernetes.io/projected/098e566e-0737-4066-bb61-44a819786dca-kube-api-access-g7jdg\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.841092 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.840941 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-grpc-tls\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.841092 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.840974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.841911 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.841472 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/098e566e-0737-4066-bb61-44a819786dca-metrics-client-ca\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.844854 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.844830 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-grpc-tls\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.845252 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.845228 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.846818 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.846770 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.846935 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.846896 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.847000 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.846921 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-tls\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.847000 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.846959 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/098e566e-0737-4066-bb61-44a819786dca-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.852259 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.852235 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7jdg\" (UniqueName: \"kubernetes.io/projected/098e566e-0737-4066-bb61-44a819786dca-kube-api-access-g7jdg\") pod \"thanos-querier-54c44dc67-kpmp5\" (UID: \"098e566e-0737-4066-bb61-44a819786dca\") " pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:16.996322 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:16.996233 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:17.015530 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:17.015475 2577 generic.go:356] "Generic (PLEG): container finished" podID="567be93d-745d-4dda-a1e2-44c77d30da63" containerID="db116e6b2ec3b6f456f29ca1926ea7e4fe9af782b64aa31d51b7194e661d7dfa" exitCode=0 Sep 15 07:26:17.015729 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:17.015587 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6z2wq" event={"ID":"567be93d-745d-4dda-a1e2-44c77d30da63","Type":"ContainerDied","Data":"db116e6b2ec3b6f456f29ca1926ea7e4fe9af782b64aa31d51b7194e661d7dfa"} Sep 15 07:26:17.596964 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:17.596845 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:26:17.616730 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:17.616627 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54c44dc67-kpmp5"] Sep 15 07:26:18.024406 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:18.024287 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" event={"ID":"6458032e-6e59-405f-908a-a59993c13da8","Type":"ContainerStarted","Data":"265c52bf76e8a9d6a9916461cc23f1125f7726007fe46593d93bcdbe90a97508"} Sep 15 07:26:18.024406 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:18.024372 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" event={"ID":"6458032e-6e59-405f-908a-a59993c13da8","Type":"ContainerStarted","Data":"b9a4d1ea308d4a027e5067b6d9b57c29b5e3ba9a9844fa4d00672d1062c9b5d4"} Sep 15 07:26:18.026985 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:18.026954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6z2wq" event={"ID":"567be93d-745d-4dda-a1e2-44c77d30da63","Type":"ContainerStarted","Data":"2a3c9680daad941b7561d7f7d2744217c0ba94baf3b066d3cd79b5f8a7f74383"} Sep 15 07:26:18.027128 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:18.027012 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6z2wq" event={"ID":"567be93d-745d-4dda-a1e2-44c77d30da63","Type":"ContainerStarted","Data":"a144a93a9cd5e37a276c20349fdc898d3ea6c6b454c3720b9a799c0f0f8ea90a"} Sep 15 07:26:18.029298 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:18.029271 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" event={"ID":"660ea539-a67b-4401-8041-b5cc09913ce0","Type":"ContainerStarted","Data":"0f452cc79211f00f0cbebfe5d765a751ecda4f8ef908bb2a3871a16f2d50ead3"} Sep 15 07:26:18.047661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:18.047615 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6z2wq" podStartSLOduration=2.813567523 podStartE2EDuration="4.047603221s" totalImagesPullingTime="1.153022299s" totalInitContainerRuntime="81.013399ms" isStatefulPod=true podCreationTimestamp="2026-09-15 07:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:18.046345052 +0000 UTC m=+69.868816459" watchObservedRunningTime="2026-09-15 07:26:18.047603221 +0000 UTC m=+69.870074623" Sep 15 07:26:18.070859 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:18.070812 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-vmnlr" podStartSLOduration=2.275819114 podStartE2EDuration="4.070793716s" totalImagesPullingTime="1.794974602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:18.070182788 +0000 UTC m=+69.892654204" watchObservedRunningTime="2026-09-15 07:26:18.070793716 +0000 UTC m=+69.893265123" Sep 15 07:26:18.541706 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:18.541671 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098e566e_0737_4066_bb61_44a819786dca.slice/crio-44854bb491a1b92fe9d78f7bfba33699b424fe0645ec885c7116ece164d1e9a9 WatchSource:0}: Error finding container 44854bb491a1b92fe9d78f7bfba33699b424fe0645ec885c7116ece164d1e9a9: Status 404 returned error can't find the container with id 44854bb491a1b92fe9d78f7bfba33699b424fe0645ec885c7116ece164d1e9a9 Sep 15 07:26:18.542214 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:18.542198 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cbd8ed_514f_4240_9bd2_f9ccd6c88930.slice/crio-258b0445f85153e43d028d7076f3822a7f3e2634aef7a6ee128dca812bbebe05 WatchSource:0}: Error finding container 258b0445f85153e43d028d7076f3822a7f3e2634aef7a6ee128dca812bbebe05: Status 404 returned error can't find the container with id 258b0445f85153e43d028d7076f3822a7f3e2634aef7a6ee128dca812bbebe05 Sep 15 07:26:18.970356 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:18.970323 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fhqmn" Sep 15 07:26:19.039944 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.039847 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerStarted","Data":"258b0445f85153e43d028d7076f3822a7f3e2634aef7a6ee128dca812bbebe05"} Sep 15 07:26:19.042838 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.042650 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" event={"ID":"098e566e-0737-4066-bb61-44a819786dca","Type":"ContainerStarted","Data":"44854bb491a1b92fe9d78f7bfba33699b424fe0645ec885c7116ece164d1e9a9"} Sep 15 07:26:19.277380 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.277302 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-757d66f764-ms55t"] Sep 15 07:26:19.280594 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.280477 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.281095 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.280754 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b"] Sep 15 07:26:19.283511 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.283172 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:19.283511 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.283387 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 15 07:26:19.283682 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.283633 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-75k9o0knforq5\"" Sep 15 07:26:19.285415 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.285018 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 15 07:26:19.285415 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.285043 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 15 07:26:19.285415 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.285254 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 15 07:26:19.285415 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.285301 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7sfsd\"" Sep 15 07:26:19.286142 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.285874 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 15 07:26:19.286778 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.286688 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zjxgj\"" Sep 15 07:26:19.294678 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.294658 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b"] Sep 15 07:26:19.303840 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.303787 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-757d66f764-ms55t"] Sep 15 07:26:19.367168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.366765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-metrics-server-audit-profiles\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.367168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.366823 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-secret-metrics-server-tls\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.367168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.366852 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sffr7\" (UniqueName: \"kubernetes.io/projected/9e5538f3-f1d8-4212-ac70-3f0a85d68f51-kube-api-access-sffr7\") pod \"monitoring-plugin-5d85778b9d-4v25b\" (UID: \"9e5538f3-f1d8-4212-ac70-3f0a85d68f51\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:19.367168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.366891 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9e5538f3-f1d8-4212-ac70-3f0a85d68f51-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-4v25b\" (UID: \"9e5538f3-f1d8-4212-ac70-3f0a85d68f51\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:19.367168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.366941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-secret-metrics-server-client-certs\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.367168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.366986 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5t4f\" (UniqueName: \"kubernetes.io/projected/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-kube-api-access-c5t4f\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.367168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.367019 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-client-ca-bundle\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.367168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.367065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.367168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.367101 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-audit-log\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.467994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9e5538f3-f1d8-4212-ac70-3f0a85d68f51-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-4v25b\" (UID: \"9e5538f3-f1d8-4212-ac70-3f0a85d68f51\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.468069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-secret-metrics-server-client-certs\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.468114 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5t4f\" (UniqueName: \"kubernetes.io/projected/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-kube-api-access-c5t4f\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.468148 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-client-ca-bundle\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.468184 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.468218 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-audit-log\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.468258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-metrics-server-audit-profiles\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.468288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-secret-metrics-server-tls\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.468313 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sffr7\" (UniqueName: \"kubernetes.io/projected/9e5538f3-f1d8-4212-ac70-3f0a85d68f51-kube-api-access-sffr7\") pod \"monitoring-plugin-5d85778b9d-4v25b\" (UID: \"9e5538f3-f1d8-4212-ac70-3f0a85d68f51\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:26:19.468760 2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 15 07:26:19.469426 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:26:19.468837 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5538f3-f1d8-4212-ac70-3f0a85d68f51-monitoring-plugin-cert podName:9e5538f3-f1d8-4212-ac70-3f0a85d68f51 nodeName:}" failed. No retries permitted until 2026-09-15 07:26:19.968816905 +0000 UTC m=+71.791288294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/9e5538f3-f1d8-4212-ac70-3f0a85d68f51-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-4v25b" (UID: "9e5538f3-f1d8-4212-ac70-3f0a85d68f51") : secret "monitoring-plugin-cert" not found Sep 15 07:26:19.470742 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.470689 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-audit-log\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.471364 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.471255 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-metrics-server-audit-profiles\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.471682 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.471634 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.472647 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.472602 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-secret-metrics-server-client-certs\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.473299 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.473276 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-client-ca-bundle\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.474227 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.474204 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-secret-metrics-server-tls\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.479320 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.479250 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5t4f\" (UniqueName: \"kubernetes.io/projected/fae5aec8-2fe9-471f-9610-72bbeea8d1bf-kube-api-access-c5t4f\") pod \"metrics-server-757d66f764-ms55t\" (UID: \"fae5aec8-2fe9-471f-9610-72bbeea8d1bf\") " pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.484050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.484009 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sffr7\" (UniqueName: \"kubernetes.io/projected/9e5538f3-f1d8-4212-ac70-3f0a85d68f51-kube-api-access-sffr7\") pod \"monitoring-plugin-5d85778b9d-4v25b\" (UID: \"9e5538f3-f1d8-4212-ac70-3f0a85d68f51\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:19.596808 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.596671 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:19.973305 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.973267 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9e5538f3-f1d8-4212-ac70-3f0a85d68f51-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-4v25b\" (UID: \"9e5538f3-f1d8-4212-ac70-3f0a85d68f51\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:19.979544 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:19.979465 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9e5538f3-f1d8-4212-ac70-3f0a85d68f51-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-4v25b\" (UID: \"9e5538f3-f1d8-4212-ac70-3f0a85d68f51\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:20.205532 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.205468 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:20.914167 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.913570 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 07:26:20.918449 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.918420 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.925566 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.925539 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 15 07:26:20.926763 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.925825 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 15 07:26:20.926763 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.926001 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 15 07:26:20.926763 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.926169 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 15 07:26:20.935974 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.935942 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.936219 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-t99qi918frd5\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.936292 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.936603 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sxp5v\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.936222 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.937082 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.937272 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.937831 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.938008 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.938104 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.938381 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 15 07:26:20.938761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.938470 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 15 07:26:20.949833 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.949810 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 07:26:20.990318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990274 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990326 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990458 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990502 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990537 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990583 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990606 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990641 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990670 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990697 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990722 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990757 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.990793 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.991463 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990817 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-config\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.991463 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.990957 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.991463 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.991029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.991463 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.991055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsd57\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-kube-api-access-xsd57\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:20.991463 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:20.991098 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092271 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092319 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-config\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092509 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092535 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsd57\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-kube-api-access-xsd57\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092588 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092631 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092689 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092724 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.093242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092774 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.094185 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.094185 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.094185 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.092877 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.094952 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.094534 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.096581 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.095351 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.096581 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.095379 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.096581 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.095426 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.096581 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.096015 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.098717 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.098166 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.098717 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.098622 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.098717 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.098677 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.099463 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.099439 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.101138 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.101112 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.101625 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.101602 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.103889 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.103213 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.106627 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.106265 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.107580 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.107540 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-config\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.107925 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.107869 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.108306 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.108269 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.121143 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.121078 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.121143 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.121081 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.121143 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.121119 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.122901 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.122863 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsd57\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-kube-api-access-xsd57\") pod \"prometheus-k8s-0\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:21.250736 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:21.250698 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:22.453184 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:22.453120 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-757d66f764-ms55t"] Sep 15 07:26:22.883209 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:22.883170 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae5aec8_2fe9_471f_9610_72bbeea8d1bf.slice/crio-a39f8e62a6bbf58c4296be906352e1cc1345c4b5401c8a62ea44b5f19140ae09 WatchSource:0}: Error finding container a39f8e62a6bbf58c4296be906352e1cc1345c4b5401c8a62ea44b5f19140ae09: Status 404 returned error can't find the container with id a39f8e62a6bbf58c4296be906352e1cc1345c4b5401c8a62ea44b5f19140ae09 Sep 15 07:26:23.063828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:23.063792 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-757d66f764-ms55t" event={"ID":"fae5aec8-2fe9-471f-9610-72bbeea8d1bf","Type":"ContainerStarted","Data":"a39f8e62a6bbf58c4296be906352e1cc1345c4b5401c8a62ea44b5f19140ae09"} Sep 15 07:26:23.163687 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:23.163622 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b"] Sep 15 07:26:23.175079 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:23.175048 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 07:26:23.391343 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:23.391165 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b196be3_f862_4158_926d_4c2aa24f4c0b.slice/crio-cad58398257535e93d9421d9fffe3e396d93a6a4010a0ddee4e61e439b96f4e5 WatchSource:0}: Error finding container cad58398257535e93d9421d9fffe3e396d93a6a4010a0ddee4e61e439b96f4e5: Status 404 returned error can't find the container with id cad58398257535e93d9421d9fffe3e396d93a6a4010a0ddee4e61e439b96f4e5 Sep 15 07:26:23.392417 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:26:23.392390 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5538f3_f1d8_4212_ac70_3f0a85d68f51.slice/crio-877588d084b75e7175bd8dcd4a437d7a9a6236f1bb38cd816accc172425b6f74 WatchSource:0}: Error finding container 877588d084b75e7175bd8dcd4a437d7a9a6236f1bb38cd816accc172425b6f74: Status 404 returned error can't find the container with id 877588d084b75e7175bd8dcd4a437d7a9a6236f1bb38cd816accc172425b6f74 Sep 15 07:26:24.080754 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.080722 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f9mmh" event={"ID":"7ef119f1-8070-4d2a-b511-1fd19295935b","Type":"ContainerStarted","Data":"787f7301cfa7276773179aa8f38095fc9e37e99ba6fa0af58aa25cb76e6500de"} Sep 15 07:26:24.085725 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.084527 2577 generic.go:356] "Generic (PLEG): container finished" podID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerID="c18499feaf77a6511667fb3ed390e0921adbc60061f22e4fafb4e818b330a14b" exitCode=0 Sep 15 07:26:24.085725 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.084616 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerDied","Data":"c18499feaf77a6511667fb3ed390e0921adbc60061f22e4fafb4e818b330a14b"} Sep 15 07:26:24.085725 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.084679 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerStarted","Data":"cad58398257535e93d9421d9fffe3e396d93a6a4010a0ddee4e61e439b96f4e5"} Sep 15 07:26:24.089100 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.088471 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m5skf" event={"ID":"735140f1-e909-449e-8129-be3752b71965","Type":"ContainerStarted","Data":"50a3fdc0039099e4811e9226100c76d739f9766ef0f24e88427dfa16159b66de"} Sep 15 07:26:24.089100 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.088986 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:26:24.094014 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.093992 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" event={"ID":"098e566e-0737-4066-bb61-44a819786dca","Type":"ContainerStarted","Data":"b87d74f0fefc665627b1bd9fa56998cbe3a602fcbec6d72b275f1d5dc149ce13"} Sep 15 07:26:24.094183 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.094167 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" event={"ID":"098e566e-0737-4066-bb61-44a819786dca","Type":"ContainerStarted","Data":"e480a4421db0f5076e7632916c87680e9ff5ab7aeae5e171952a5d9f2f47cc84"} Sep 15 07:26:24.098462 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.098422 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" event={"ID":"9e5538f3-f1d8-4212-ac70-3f0a85d68f51","Type":"ContainerStarted","Data":"877588d084b75e7175bd8dcd4a437d7a9a6236f1bb38cd816accc172425b6f74"} Sep 15 07:26:24.104615 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.103974 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" event={"ID":"6458032e-6e59-405f-908a-a59993c13da8","Type":"ContainerStarted","Data":"b5b33f86cff944a5a7c517dee55b6bc37c61f1715215820edf8b537b3e97e560"} Sep 15 07:26:24.108038 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.107564 2577 generic.go:356] "Generic (PLEG): container finished" podID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerID="ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102" exitCode=0 Sep 15 07:26:24.108038 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.107649 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerDied","Data":"ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102"} Sep 15 07:26:24.113736 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.111383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657ccdbdf6-22hqp" event={"ID":"d18b7df9-0525-4691-afc3-e420e069a100","Type":"ContainerStarted","Data":"e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce"} Sep 15 07:26:24.115530 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.115112 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v8tt9" event={"ID":"e623486b-2e21-4cfc-844d-25fefc0f4418","Type":"ContainerStarted","Data":"33fc969ddb42e617699d8b73f5a3bc7208586719139500e7a3b87d24c99d07a9"} Sep 15 07:26:24.141724 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.141127 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m5skf" podStartSLOduration=67.986931067 podStartE2EDuration="1m15.141107626s" totalImagesPullingTime="7.154176559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:24.14070985 +0000 UTC m=+75.963181256" watchObservedRunningTime="2026-09-15 07:26:24.141107626 +0000 UTC m=+75.963579026" Sep 15 07:26:24.187506 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.186637 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-v8tt9" podStartSLOduration=67.008508812 podStartE2EDuration="1m15.18661589s" totalImagesPullingTime="8.178107078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:24.186438297 +0000 UTC m=+76.008909703" watchObservedRunningTime="2026-09-15 07:26:24.18661589 +0000 UTC m=+76.009087295" Sep 15 07:26:24.188759 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.187939 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-657ccdbdf6-22hqp" podStartSLOduration=2.359292393 podStartE2EDuration="10.18792703s" totalImagesPullingTime="7.828634637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:24.167643249 +0000 UTC m=+75.990114655" watchObservedRunningTime="2026-09-15 07:26:24.18792703 +0000 UTC m=+76.010398435" Sep 15 07:26:24.207661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.207597 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-f9qt5" podStartSLOduration=8.325417262 podStartE2EDuration="10.207575091s" totalImagesPullingTime="1.882157829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:24.207347381 +0000 UTC m=+76.029818786" watchObservedRunningTime="2026-09-15 07:26:24.207575091 +0000 UTC m=+76.030046497" Sep 15 07:26:24.750066 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.749903 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:24.750066 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.749971 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:24.755380 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:24.755351 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:25.127426 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:25.127310 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f9mmh" event={"ID":"7ef119f1-8070-4d2a-b511-1fd19295935b","Type":"ContainerStarted","Data":"1c7229953ee66ab0856132cab803b34980ecfd3324778310add42730299584ea"} Sep 15 07:26:25.132839 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:25.132527 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" event={"ID":"098e566e-0737-4066-bb61-44a819786dca","Type":"ContainerStarted","Data":"dab5f5323d39c79d9eedaa6d52095bf139a4cbc3f4fed9218119a7bf2ed610ca"} Sep 15 07:26:25.139224 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:25.139165 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:26:25.172728 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:25.172415 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f9mmh" podStartSLOduration=67.467927962 podStartE2EDuration="1m16.172393157s" totalImagesPullingTime="8.704465195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:25.148708454 +0000 UTC m=+76.971179869" watchObservedRunningTime="2026-09-15 07:26:25.172393157 +0000 UTC m=+76.994864545" Sep 15 07:26:26.139650 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:26.139608 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-757d66f764-ms55t" event={"ID":"fae5aec8-2fe9-471f-9610-72bbeea8d1bf","Type":"ContainerStarted","Data":"73b258342ced46386345abe9465da95e74eca990eac470100584c334e788a641"} Sep 15 07:26:26.142096 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:26.142033 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" event={"ID":"9e5538f3-f1d8-4212-ac70-3f0a85d68f51","Type":"ContainerStarted","Data":"09f1558ecf2752ad5a669dbc8bb72e54b29b18fb72734621829da6d81d6e8852"} Sep 15 07:26:26.142416 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:26.142395 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:26.149339 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:26.149318 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" Sep 15 07:26:26.160982 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:26.160929 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-757d66f764-ms55t" podStartSLOduration=4.714204283 podStartE2EDuration="7.16091018s" totalImagesPullingTime="2.446705897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:26.160210265 +0000 UTC m=+77.982681672" watchObservedRunningTime="2026-09-15 07:26:26.16091018 +0000 UTC m=+77.983381587" Sep 15 07:26:26.182519 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:26.182154 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-4v25b" podStartSLOduration=5.253017932 podStartE2EDuration="7.182135673s" totalImagesPullingTime="1.929117741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:26.179680405 +0000 UTC m=+78.002151810" watchObservedRunningTime="2026-09-15 07:26:26.182135673 +0000 UTC m=+78.004607079" Sep 15 07:26:28.161013 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:28.160835 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerStarted","Data":"194f4b0a13dcef429305f4a6ab32bf4640a5b0f39d0e868df12a108108f5be0c"} Sep 15 07:26:28.161013 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:28.160914 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerStarted","Data":"11bc95a4c081be9ddcf8b453b0181c0e830d9df3d57345febc8ce0ab2ad9edf8"} Sep 15 07:26:28.161013 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:28.160955 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerStarted","Data":"9342028d4930dc413ea83f61915536c0cff49f05a616abba007b93b377153dc5"} Sep 15 07:26:29.170516 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.169968 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerStarted","Data":"631b045bc27fcdb9d88852e030375dbc260d5af8c4641914b62133dc78f0e5f5"} Sep 15 07:26:29.170516 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.170089 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerStarted","Data":"a7c902e34672bac1b67acf888bdc5ce8f412c6ee5534473736c41ad6e0c374b5"} Sep 15 07:26:29.170516 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.170133 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerStarted","Data":"4adc8920e921cf14b994aaf4fddb3e5b214bfc33e2cfce6e9f5cfc35440cab4b"} Sep 15 07:26:29.173646 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.173604 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" event={"ID":"098e566e-0737-4066-bb61-44a819786dca","Type":"ContainerStarted","Data":"81d146829ed0d355110016e3c1eb10ff673a6981beacdb5352818e3f4fc5f6ff"} Sep 15 07:26:29.173706 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.173657 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" event={"ID":"098e566e-0737-4066-bb61-44a819786dca","Type":"ContainerStarted","Data":"aac3baf57575c5a0e722f3426251ba09265426bb798735d0f5cfbdd9b486d655"} Sep 15 07:26:29.173706 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.173697 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" event={"ID":"098e566e-0737-4066-bb61-44a819786dca","Type":"ContainerStarted","Data":"7851805be9432b7656b6f2ef907a58336f38c68c57ae826b79924c8ebca71050"} Sep 15 07:26:29.174112 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.174035 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:29.181107 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.181084 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" Sep 15 07:26:29.213682 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.213625 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.001503621 podStartE2EDuration="9.213604543s" totalImagesPullingTime="4.192139859s" totalInitContainerRuntime="19.961063ms" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:29.20986712 +0000 UTC m=+81.032338549" watchObservedRunningTime="2026-09-15 07:26:29.213604543 +0000 UTC m=+81.036075948" Sep 15 07:26:29.241929 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:29.241871 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-54c44dc67-kpmp5" podStartSLOduration=3.7679360170000002 podStartE2EDuration="13.241854896s" totalImagesPullingTime="9.473918879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:29.241114415 +0000 UTC m=+81.063585843" watchObservedRunningTime="2026-09-15 07:26:29.241854896 +0000 UTC m=+81.064326302" Sep 15 07:26:30.183519 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:30.183461 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerStarted","Data":"42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67"} Sep 15 07:26:30.183951 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:30.183569 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerStarted","Data":"f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264"} Sep 15 07:26:30.183951 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:30.183613 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerStarted","Data":"0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7"} Sep 15 07:26:30.183951 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:30.183644 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerStarted","Data":"d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f"} Sep 15 07:26:30.183951 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:30.183674 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerStarted","Data":"72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa"} Sep 15 07:26:30.183951 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:30.183700 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerStarted","Data":"f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a"} Sep 15 07:26:30.218394 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:30.218340 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.948332872 podStartE2EDuration="15.218322387s" totalImagesPullingTime="10.248236819s" totalInitContainerRuntime="21.752696ms" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:30.213477558 +0000 UTC m=+82.035949001" watchObservedRunningTime="2026-09-15 07:26:30.218322387 +0000 UTC m=+82.040793791" Sep 15 07:26:31.251047 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:31.250929 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:26:34.771296 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:34.771265 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-657ccdbdf6-22hqp"] Sep 15 07:26:38.223177 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:38.223128 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-vzq75" event={"ID":"9bf86a8e-64a0-4edf-be9b-1543f8ab172a","Type":"ContainerStarted","Data":"cfc3887931321d66d76fcbb4808cf71c208c7f277b1e08272ccdf444dca717be"} Sep 15 07:26:38.223724 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:38.223328 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-vzq75" Sep 15 07:26:38.236563 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:38.236533 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-vzq75" Sep 15 07:26:38.247839 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:38.247782 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-vzq75" podStartSLOduration=2.09790829 podStartE2EDuration="29.247761945s" totalImagesPullingTime="27.149853655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:26:38.245762932 +0000 UTC m=+90.068234337" watchObservedRunningTime="2026-09-15 07:26:38.247761945 +0000 UTC m=+90.070233353" Sep 15 07:26:39.597094 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:39.597051 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:39.597579 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:39.597157 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:44.587276 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:44.587233 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6msdh_f8933686-b777-4397-bcaa-b0ce3f7f4a9e/serve-healthcheck-canary/0.log" Sep 15 07:26:56.144397 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:56.144363 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m5skf" Sep 15 07:26:59.602540 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:59.602502 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:59.606307 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:59.606285 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-757d66f764-ms55t" Sep 15 07:26:59.793968 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:26:59.793892 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-657ccdbdf6-22hqp" podUID="d18b7df9-0525-4691-afc3-e420e069a100" containerName="console" containerID="cri-o://e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce" gracePeriod=15 Sep 15 07:27:00.089296 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.089264 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:27:00.148143 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.148106 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-console-config\") pod \"d18b7df9-0525-4691-afc3-e420e069a100\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " Sep 15 07:27:00.148333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.148158 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-oauth-serving-cert\") pod \"d18b7df9-0525-4691-afc3-e420e069a100\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " Sep 15 07:27:00.148333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.148175 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-service-ca\") pod \"d18b7df9-0525-4691-afc3-e420e069a100\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " Sep 15 07:27:00.148333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.148218 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vhbc\" (UniqueName: \"kubernetes.io/projected/d18b7df9-0525-4691-afc3-e420e069a100-kube-api-access-7vhbc\") pod \"d18b7df9-0525-4691-afc3-e420e069a100\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " Sep 15 07:27:00.148333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.148250 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-serving-cert\") pod \"d18b7df9-0525-4691-afc3-e420e069a100\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " Sep 15 07:27:00.148333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.148277 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-oauth-config\") pod \"d18b7df9-0525-4691-afc3-e420e069a100\" (UID: \"d18b7df9-0525-4691-afc3-e420e069a100\") " Sep 15 07:27:00.148681 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.148644 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-console-config" (OuterVolumeSpecName: "console-config") pod "d18b7df9-0525-4691-afc3-e420e069a100" (UID: "d18b7df9-0525-4691-afc3-e420e069a100"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:00.148797 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.148595 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-service-ca" (OuterVolumeSpecName: "service-ca") pod "d18b7df9-0525-4691-afc3-e420e069a100" (UID: "d18b7df9-0525-4691-afc3-e420e069a100"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:00.148849 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.148803 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d18b7df9-0525-4691-afc3-e420e069a100" (UID: "d18b7df9-0525-4691-afc3-e420e069a100"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:00.150626 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.150598 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d18b7df9-0525-4691-afc3-e420e069a100" (UID: "d18b7df9-0525-4691-afc3-e420e069a100"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:00.150739 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.150693 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d18b7df9-0525-4691-afc3-e420e069a100-kube-api-access-7vhbc" (OuterVolumeSpecName: "kube-api-access-7vhbc") pod "d18b7df9-0525-4691-afc3-e420e069a100" (UID: "d18b7df9-0525-4691-afc3-e420e069a100"). InnerVolumeSpecName "kube-api-access-7vhbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:27:00.150739 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.150707 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d18b7df9-0525-4691-afc3-e420e069a100" (UID: "d18b7df9-0525-4691-afc3-e420e069a100"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:00.249770 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.249667 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-oauth-config\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:00.249770 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.249714 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-console-config\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:00.249770 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.249724 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-oauth-serving-cert\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:00.249770 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.249733 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d18b7df9-0525-4691-afc3-e420e069a100-service-ca\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:00.249770 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.249741 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vhbc\" (UniqueName: \"kubernetes.io/projected/d18b7df9-0525-4691-afc3-e420e069a100-kube-api-access-7vhbc\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:00.249770 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.249750 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d18b7df9-0525-4691-afc3-e420e069a100-console-serving-cert\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:00.323725 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.323686 2577 generic.go:356] "Generic (PLEG): container finished" podID="d18b7df9-0525-4691-afc3-e420e069a100" containerID="e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce" exitCode=0 Sep 15 07:27:00.323923 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.323750 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-657ccdbdf6-22hqp" Sep 15 07:27:00.323923 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.323829 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657ccdbdf6-22hqp" event={"ID":"d18b7df9-0525-4691-afc3-e420e069a100","Type":"ContainerDied","Data":"e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce"} Sep 15 07:27:00.323923 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.323902 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-657ccdbdf6-22hqp" event={"ID":"d18b7df9-0525-4691-afc3-e420e069a100","Type":"ContainerDied","Data":"306a5771beccb6b92a91a0185269f832af83374c090b4efc3eacee1b0910f66b"} Sep 15 07:27:00.324094 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.323938 2577 scope.go:117] "RemoveContainer" containerID="e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce" Sep 15 07:27:00.335871 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.335827 2577 scope.go:117] "RemoveContainer" containerID="e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce" Sep 15 07:27:00.336175 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:27:00.336138 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce\": container with ID starting with e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce not found: ID does not exist" containerID="e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce" Sep 15 07:27:00.336268 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.336184 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce"} err="failed to get container status \"e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce\": rpc error: code = NotFound desc = could not find container \"e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce\": container with ID starting with e92b8629b1694cbe8fd32a6c51774be31dee394541e32255a01346d60cf6b0ce not found: ID does not exist" Sep 15 07:27:00.350009 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.349971 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-657ccdbdf6-22hqp"] Sep 15 07:27:00.353872 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.353840 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-657ccdbdf6-22hqp"] Sep 15 07:27:00.690089 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:00.690060 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d18b7df9-0525-4691-afc3-e420e069a100" path="/var/lib/kubelet/pods/d18b7df9-0525-4691-afc3-e420e069a100/volumes" Sep 15 07:27:21.251109 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:21.251068 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:21.324347 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:21.324320 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:21.426860 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:21.426835 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:34.911931 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:34.911895 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:27:34.912316 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:34.912167 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="alertmanager" containerID="cri-o://f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a" gracePeriod=120 Sep 15 07:27:34.912316 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:34.912245 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy-metric" containerID="cri-o://f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264" gracePeriod=120 Sep 15 07:27:34.912316 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:34.912266 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="prom-label-proxy" containerID="cri-o://42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67" gracePeriod=120 Sep 15 07:27:34.912509 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:34.912304 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="config-reloader" containerID="cri-o://72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa" gracePeriod=120 Sep 15 07:27:34.912509 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:34.912323 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy" containerID="cri-o://0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7" gracePeriod=120 Sep 15 07:27:34.912509 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:34.912283 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy-web" containerID="cri-o://d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f" gracePeriod=120 Sep 15 07:27:35.470920 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.470888 2577 generic.go:356] "Generic (PLEG): container finished" podID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerID="42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67" exitCode=0 Sep 15 07:27:35.470920 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.470915 2577 generic.go:356] "Generic (PLEG): container finished" podID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerID="f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264" exitCode=0 Sep 15 07:27:35.470920 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.470923 2577 generic.go:356] "Generic (PLEG): container finished" podID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerID="0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7" exitCode=0 Sep 15 07:27:35.470920 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.470929 2577 generic.go:356] "Generic (PLEG): container finished" podID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerID="72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa" exitCode=0 Sep 15 07:27:35.471181 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.470934 2577 generic.go:356] "Generic (PLEG): container finished" podID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerID="f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a" exitCode=0 Sep 15 07:27:35.471181 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.470955 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerDied","Data":"42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67"} Sep 15 07:27:35.471181 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.471020 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerDied","Data":"f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264"} Sep 15 07:27:35.471181 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.471051 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerDied","Data":"0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7"} Sep 15 07:27:35.471181 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.471078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerDied","Data":"72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa"} Sep 15 07:27:35.471181 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:35.471101 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerDied","Data":"f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a"} Sep 15 07:27:36.152174 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.152147 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.254352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254257 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-metrics-client-ca\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.254352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254310 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-web\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.254611 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254511 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-out\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.254611 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254599 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-web-config\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.254728 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254605 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:36.254728 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254625 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.254728 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254653 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-tls-assets\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.254728 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254678 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-metric\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.254728 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254705 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lc7ft\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-kube-api-access-lc7ft\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.255144 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254738 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-main-db\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.255144 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254790 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-trusted-ca-bundle\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.255144 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254822 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-cluster-tls-config\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.255144 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254852 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-volume\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.255144 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.254881 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-main-tls\") pod \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\" (UID: \"99cbd8ed-514f-4240-9bd2-f9ccd6c88930\") " Sep 15 07:27:36.255144 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.255110 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-metrics-client-ca\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.255450 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.255309 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:27:36.255811 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.255782 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:36.258152 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.258124 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:27:36.258458 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.258417 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:36.258580 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.258470 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-out" (OuterVolumeSpecName: "config-out") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:27:36.258802 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.258770 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:36.259068 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.259037 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-kube-api-access-lc7ft" (OuterVolumeSpecName: "kube-api-access-lc7ft") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "kube-api-access-lc7ft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:27:36.259149 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.259099 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:36.259292 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.259271 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-volume" (OuterVolumeSpecName: "config-volume") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:36.259402 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.259379 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:36.263046 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.263018 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:36.269310 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.269290 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-web-config" (OuterVolumeSpecName: "web-config") pod "99cbd8ed-514f-4240-9bd2-f9ccd6c88930" (UID: "99cbd8ed-514f-4240-9bd2-f9ccd6c88930"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:36.356409 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356370 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-web-config\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356409 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356407 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356422 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-tls-assets\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356435 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356448 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc7ft\" (UniqueName: \"kubernetes.io/projected/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-kube-api-access-lc7ft\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356460 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-main-db\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356474 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356509 2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-cluster-tls-config\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356521 2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-volume\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356534 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-main-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356546 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.356614 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.356559 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99cbd8ed-514f-4240-9bd2-f9ccd6c88930-config-out\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:36.478604 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.478576 2577 generic.go:356] "Generic (PLEG): container finished" podID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerID="d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f" exitCode=0 Sep 15 07:27:36.478723 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.478627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerDied","Data":"d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f"} Sep 15 07:27:36.478723 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.478657 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.478723 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.478705 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99cbd8ed-514f-4240-9bd2-f9ccd6c88930","Type":"ContainerDied","Data":"258b0445f85153e43d028d7076f3822a7f3e2634aef7a6ee128dca812bbebe05"} Sep 15 07:27:36.478849 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.478737 2577 scope.go:117] "RemoveContainer" containerID="42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67" Sep 15 07:27:36.486827 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.486796 2577 scope.go:117] "RemoveContainer" containerID="f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264" Sep 15 07:27:36.493388 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.493371 2577 scope.go:117] "RemoveContainer" containerID="0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7" Sep 15 07:27:36.499747 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.499730 2577 scope.go:117] "RemoveContainer" containerID="d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f" Sep 15 07:27:36.506026 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.506007 2577 scope.go:117] "RemoveContainer" containerID="72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa" Sep 15 07:27:36.512814 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.512797 2577 scope.go:117] "RemoveContainer" containerID="f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a" Sep 15 07:27:36.519248 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.519232 2577 scope.go:117] "RemoveContainer" containerID="ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102" Sep 15 07:27:36.525609 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.525592 2577 scope.go:117] "RemoveContainer" containerID="42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67" Sep 15 07:27:36.525885 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:27:36.525867 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67\": container with ID starting with 42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67 not found: ID does not exist" containerID="42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67" Sep 15 07:27:36.525931 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.525893 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67"} err="failed to get container status \"42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67\": rpc error: code = NotFound desc = could not find container \"42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67\": container with ID starting with 42ab378e2dad750249207f8333d88c29bcb35287dc040db35779f6aa5ec6cb67 not found: ID does not exist" Sep 15 07:27:36.525931 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.525909 2577 scope.go:117] "RemoveContainer" containerID="f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264" Sep 15 07:27:36.526151 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:27:36.526131 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264\": container with ID starting with f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264 not found: ID does not exist" containerID="f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264" Sep 15 07:27:36.526244 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.526154 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264"} err="failed to get container status \"f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264\": rpc error: code = NotFound desc = could not find container \"f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264\": container with ID starting with f141883bd3e9e68d9a7672e91f7612b0571dd73febf460ebc7452b3d71e0d264 not found: ID does not exist" Sep 15 07:27:36.526244 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.526169 2577 scope.go:117] "RemoveContainer" containerID="0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7" Sep 15 07:27:36.526381 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:27:36.526363 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7\": container with ID starting with 0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7 not found: ID does not exist" containerID="0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7" Sep 15 07:27:36.526442 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.526390 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7"} err="failed to get container status \"0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7\": rpc error: code = NotFound desc = could not find container \"0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7\": container with ID starting with 0f8c4639ce31f22a6c1105e8b57718b8253c2b77f9af92faa3a48e91f0f8c9a7 not found: ID does not exist" Sep 15 07:27:36.526442 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.526412 2577 scope.go:117] "RemoveContainer" containerID="d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f" Sep 15 07:27:36.526698 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:27:36.526682 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f\": container with ID starting with d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f not found: ID does not exist" containerID="d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f" Sep 15 07:27:36.526743 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.526704 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f"} err="failed to get container status \"d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f\": rpc error: code = NotFound desc = could not find container \"d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f\": container with ID starting with d800ab8340eaadbb8850640541d2fb6b44be61cf8719831400fcc0f19cc13d6f not found: ID does not exist" Sep 15 07:27:36.526743 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.526721 2577 scope.go:117] "RemoveContainer" containerID="72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa" Sep 15 07:27:36.526947 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:27:36.526933 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa\": container with ID starting with 72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa not found: ID does not exist" containerID="72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa" Sep 15 07:27:36.526989 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.526950 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa"} err="failed to get container status \"72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa\": rpc error: code = NotFound desc = could not find container \"72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa\": container with ID starting with 72be984e45d84adfe9d7b208feec6e6eaa89c5ddf60db869e50722b3e31751fa not found: ID does not exist" Sep 15 07:27:36.526989 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.526962 2577 scope.go:117] "RemoveContainer" containerID="f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a" Sep 15 07:27:36.527158 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:27:36.527141 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a\": container with ID starting with f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a not found: ID does not exist" containerID="f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a" Sep 15 07:27:36.527201 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.527162 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a"} err="failed to get container status \"f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a\": rpc error: code = NotFound desc = could not find container \"f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a\": container with ID starting with f367215381ea4c875b3bb7d798e1c3538dde8985dbf57bcc1318f3020607ee5a not found: ID does not exist" Sep 15 07:27:36.527201 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.527183 2577 scope.go:117] "RemoveContainer" containerID="ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102" Sep 15 07:27:36.527421 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:27:36.527408 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102\": container with ID starting with ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102 not found: ID does not exist" containerID="ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102" Sep 15 07:27:36.527466 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.527424 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102"} err="failed to get container status \"ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102\": rpc error: code = NotFound desc = could not find container \"ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102\": container with ID starting with ff202db40e96dc0dbf5fb161b490dc9ae34861ed4b0fdb62ad1548c6a6f7e102 not found: ID does not exist" Sep 15 07:27:36.564341 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.564319 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:27:36.569911 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.569884 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:27:36.628125 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628103 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:27:36.628389 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628378 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="prom-label-proxy" Sep 15 07:27:36.628434 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628393 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="prom-label-proxy" Sep 15 07:27:36.628434 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628407 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy-web" Sep 15 07:27:36.628434 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628412 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy-web" Sep 15 07:27:36.628434 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628420 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy" Sep 15 07:27:36.628434 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628426 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628436 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy-metric" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628442 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy-metric" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628449 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18b7df9-0525-4691-afc3-e420e069a100" containerName="console" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628454 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18b7df9-0525-4691-afc3-e420e069a100" containerName="console" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628462 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="alertmanager" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628467 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="alertmanager" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628473 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="config-reloader" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628478 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="config-reloader" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628504 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="init-config-reloader" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628509 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="init-config-reloader" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628551 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy-web" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628558 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy-metric" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628563 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="kube-rbac-proxy" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628570 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="alertmanager" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628577 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d18b7df9-0525-4691-afc3-e420e069a100" containerName="console" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628581 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="prom-label-proxy" Sep 15 07:27:36.628599 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.628588 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" containerName="config-reloader" Sep 15 07:27:36.632262 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.632246 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.635450 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.635431 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 15 07:27:36.635562 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.635498 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 15 07:27:36.635562 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.635519 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 15 07:27:36.635681 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.635558 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 15 07:27:36.635681 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.635598 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 15 07:27:36.635681 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.635655 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 15 07:27:36.635828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.635753 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 15 07:27:36.635828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.635818 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 15 07:27:36.636198 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.636180 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j46r7\"" Sep 15 07:27:36.640694 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.640677 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 15 07:27:36.652591 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.652562 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:27:36.658332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658307 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/847acd2b-7afe-499c-94ae-8a165bfdfa0b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658436 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658341 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-config-volume\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658436 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658436 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658385 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658625 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658455 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658625 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658501 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658625 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658523 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658625 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/847acd2b-7afe-499c-94ae-8a165bfdfa0b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658625 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/847acd2b-7afe-499c-94ae-8a165bfdfa0b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658875 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658633 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/847acd2b-7afe-499c-94ae-8a165bfdfa0b-config-out\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658875 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-web-config\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658875 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658675 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/847acd2b-7afe-499c-94ae-8a165bfdfa0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.658875 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.658703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvqzb\" (UniqueName: \"kubernetes.io/projected/847acd2b-7afe-499c-94ae-8a165bfdfa0b-kube-api-access-bvqzb\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.689492 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.689454 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99cbd8ed-514f-4240-9bd2-f9ccd6c88930" path="/var/lib/kubelet/pods/99cbd8ed-514f-4240-9bd2-f9ccd6c88930/volumes" Sep 15 07:27:36.759683 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.759628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.759683 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.759655 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/847acd2b-7afe-499c-94ae-8a165bfdfa0b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.759683 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.759680 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/847acd2b-7afe-499c-94ae-8a165bfdfa0b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.759851 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.759730 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/847acd2b-7afe-499c-94ae-8a165bfdfa0b-config-out\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.759851 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.759771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-web-config\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.759851 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.759807 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/847acd2b-7afe-499c-94ae-8a165bfdfa0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.759851 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.759846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvqzb\" (UniqueName: \"kubernetes.io/projected/847acd2b-7afe-499c-94ae-8a165bfdfa0b-kube-api-access-bvqzb\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.760050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.759879 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/847acd2b-7afe-499c-94ae-8a165bfdfa0b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.760104 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.760056 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/847acd2b-7afe-499c-94ae-8a165bfdfa0b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.760157 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.760102 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-config-volume\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.760157 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.760137 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.760261 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.760164 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.760261 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.760205 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.760261 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.760234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.760609 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.760577 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/847acd2b-7afe-499c-94ae-8a165bfdfa0b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.761192 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.760857 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/847acd2b-7afe-499c-94ae-8a165bfdfa0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.762773 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.762731 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/847acd2b-7afe-499c-94ae-8a165bfdfa0b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.762773 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.762740 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.763056 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.762968 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/847acd2b-7afe-499c-94ae-8a165bfdfa0b-config-out\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.763056 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.763016 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-web-config\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.763205 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.763142 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.763258 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.763238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.763355 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.763337 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-config-volume\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.763747 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.763729 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.764441 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.764426 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/847acd2b-7afe-499c-94ae-8a165bfdfa0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.768894 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.768876 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvqzb\" (UniqueName: \"kubernetes.io/projected/847acd2b-7afe-499c-94ae-8a165bfdfa0b-kube-api-access-bvqzb\") pod \"alertmanager-main-0\" (UID: \"847acd2b-7afe-499c-94ae-8a165bfdfa0b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:36.941644 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:36.941620 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:27:37.221810 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:37.221781 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:27:37.222707 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:27:37.222668 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847acd2b_7afe_499c_94ae_8a165bfdfa0b.slice/crio-229aa62aed6b063286d639ec498ffa950e2bb583e7ea4e97589a93f0a1b1c74b WatchSource:0}: Error finding container 229aa62aed6b063286d639ec498ffa950e2bb583e7ea4e97589a93f0a1b1c74b: Status 404 returned error can't find the container with id 229aa62aed6b063286d639ec498ffa950e2bb583e7ea4e97589a93f0a1b1c74b Sep 15 07:27:37.484778 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:37.484695 2577 generic.go:356] "Generic (PLEG): container finished" podID="847acd2b-7afe-499c-94ae-8a165bfdfa0b" containerID="7180dd9e6a1a418cc8a0330037db55c28f92b6b233b1509c5afd78d2ee044d29" exitCode=0 Sep 15 07:27:37.484935 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:37.484789 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"847acd2b-7afe-499c-94ae-8a165bfdfa0b","Type":"ContainerDied","Data":"7180dd9e6a1a418cc8a0330037db55c28f92b6b233b1509c5afd78d2ee044d29"} Sep 15 07:27:37.484935 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:37.484875 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"847acd2b-7afe-499c-94ae-8a165bfdfa0b","Type":"ContainerStarted","Data":"229aa62aed6b063286d639ec498ffa950e2bb583e7ea4e97589a93f0a1b1c74b"} Sep 15 07:27:38.493445 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.493412 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"847acd2b-7afe-499c-94ae-8a165bfdfa0b","Type":"ContainerStarted","Data":"4b9c913f2af1c48f55819cc9857c0204951f59369cfcd83d570c542a2e9ddf44"} Sep 15 07:27:38.493838 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.493520 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"847acd2b-7afe-499c-94ae-8a165bfdfa0b","Type":"ContainerStarted","Data":"b6905c0b7860a9b6c8a67c88fad0e9c10ef777f1716b5ff087d78b26b0dae71e"} Sep 15 07:27:38.493838 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.493548 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"847acd2b-7afe-499c-94ae-8a165bfdfa0b","Type":"ContainerStarted","Data":"69307104dd035d05227925df33d1f673dd478d5719918b7e13e32a9c106d2b8b"} Sep 15 07:27:38.493838 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.493573 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"847acd2b-7afe-499c-94ae-8a165bfdfa0b","Type":"ContainerStarted","Data":"614ec47397abcaff60959edb88274c3544c925f3f4a44dbccb420af49ec95a21"} Sep 15 07:27:38.493838 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.493597 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"847acd2b-7afe-499c-94ae-8a165bfdfa0b","Type":"ContainerStarted","Data":"8210432399ffd9e849c72f49b51b9e0d5c8cc6c6b67d7e52e23e4c113f597178"} Sep 15 07:27:38.493838 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.493619 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"847acd2b-7afe-499c-94ae-8a165bfdfa0b","Type":"ContainerStarted","Data":"a0363eb27d1dd011eb27d83342c778a1d762a607d66098a9d57bc6f8e57f2d65"} Sep 15 07:27:38.523023 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.522964 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.507251198 podStartE2EDuration="2.522945333s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.694135ms" isStatefulPod=false podCreationTimestamp="2026-09-15 07:27:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:27:38.521945134 +0000 UTC m=+150.344416550" watchObservedRunningTime="2026-09-15 07:27:38.522945333 +0000 UTC m=+150.345416742" Sep 15 07:27:38.988800 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.988763 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw"] Sep 15 07:27:38.991918 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.991892 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:38.994883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.994855 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 15 07:27:38.994883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.994855 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 15 07:27:38.995074 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.994913 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 15 07:27:38.995074 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.994882 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 15 07:27:38.995195 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.995174 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-g2m6g\"" Sep 15 07:27:38.995247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:38.995228 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 15 07:27:39.000643 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.000618 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 15 07:27:39.004823 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.004796 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw"] Sep 15 07:27:39.079660 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.079627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpw7f\" (UniqueName: \"kubernetes.io/projected/efd68fd7-3686-466c-a1b1-4f237e1affc1-kube-api-access-hpw7f\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.079660 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.079673 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.079883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.079696 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-telemeter-client-tls\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.079883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.079776 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efd68fd7-3686-466c-a1b1-4f237e1affc1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.079883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.079812 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efd68fd7-3686-466c-a1b1-4f237e1affc1-serving-certs-ca-bundle\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.079883 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.079872 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efd68fd7-3686-466c-a1b1-4f237e1affc1-metrics-client-ca\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.080021 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.079921 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-secret-telemeter-client\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.080021 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.079938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-federate-client-tls\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.180715 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.180680 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efd68fd7-3686-466c-a1b1-4f237e1affc1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.180880 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.180720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efd68fd7-3686-466c-a1b1-4f237e1affc1-serving-certs-ca-bundle\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.180880 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.180784 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efd68fd7-3686-466c-a1b1-4f237e1affc1-metrics-client-ca\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.180880 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.180836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-secret-telemeter-client\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.180880 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.180859 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-federate-client-tls\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.181052 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.180921 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpw7f\" (UniqueName: \"kubernetes.io/projected/efd68fd7-3686-466c-a1b1-4f237e1affc1-kube-api-access-hpw7f\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.181052 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.180978 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.181052 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.181023 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-telemeter-client-tls\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.181596 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.181571 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efd68fd7-3686-466c-a1b1-4f237e1affc1-serving-certs-ca-bundle\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.181714 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.181671 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efd68fd7-3686-466c-a1b1-4f237e1affc1-metrics-client-ca\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.181714 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.181677 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efd68fd7-3686-466c-a1b1-4f237e1affc1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.183551 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.183526 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-telemeter-client-tls\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.183674 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.183652 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-federate-client-tls\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.183775 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.183690 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.184141 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.184118 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/efd68fd7-3686-466c-a1b1-4f237e1affc1-secret-telemeter-client\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.190879 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.190856 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpw7f\" (UniqueName: \"kubernetes.io/projected/efd68fd7-3686-466c-a1b1-4f237e1affc1-kube-api-access-hpw7f\") pod \"telemeter-client-74b4cf6f8b-69ktw\" (UID: \"efd68fd7-3686-466c-a1b1-4f237e1affc1\") " pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.273318 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.273243 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 07:27:39.273669 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.273634 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="prometheus" containerID="cri-o://9342028d4930dc413ea83f61915536c0cff49f05a616abba007b93b377153dc5" gracePeriod=600 Sep 15 07:27:39.273775 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.273667 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://631b045bc27fcdb9d88852e030375dbc260d5af8c4641914b62133dc78f0e5f5" gracePeriod=600 Sep 15 07:27:39.273775 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.273674 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="thanos-sidecar" containerID="cri-o://194f4b0a13dcef429305f4a6ab32bf4640a5b0f39d0e868df12a108108f5be0c" gracePeriod=600 Sep 15 07:27:39.273775 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.273637 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy" containerID="cri-o://a7c902e34672bac1b67acf888bdc5ce8f412c6ee5534473736c41ad6e0c374b5" gracePeriod=600 Sep 15 07:27:39.273775 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.273673 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="config-reloader" containerID="cri-o://11bc95a4c081be9ddcf8b453b0181c0e830d9df3d57345febc8ce0ab2ad9edf8" gracePeriod=600 Sep 15 07:27:39.273980 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.273922 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy-web" containerID="cri-o://4adc8920e921cf14b994aaf4fddb3e5b214bfc33e2cfce6e9f5cfc35440cab4b" gracePeriod=600 Sep 15 07:27:39.302540 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.302514 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" Sep 15 07:27:39.512521 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512464 2577 generic.go:356] "Generic (PLEG): container finished" podID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerID="631b045bc27fcdb9d88852e030375dbc260d5af8c4641914b62133dc78f0e5f5" exitCode=0 Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512528 2577 generic.go:356] "Generic (PLEG): container finished" podID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerID="a7c902e34672bac1b67acf888bdc5ce8f412c6ee5534473736c41ad6e0c374b5" exitCode=0 Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512542 2577 generic.go:356] "Generic (PLEG): container finished" podID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerID="4adc8920e921cf14b994aaf4fddb3e5b214bfc33e2cfce6e9f5cfc35440cab4b" exitCode=0 Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512549 2577 generic.go:356] "Generic (PLEG): container finished" podID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerID="194f4b0a13dcef429305f4a6ab32bf4640a5b0f39d0e868df12a108108f5be0c" exitCode=0 Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512556 2577 generic.go:356] "Generic (PLEG): container finished" podID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerID="11bc95a4c081be9ddcf8b453b0181c0e830d9df3d57345febc8ce0ab2ad9edf8" exitCode=0 Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512546 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerDied","Data":"631b045bc27fcdb9d88852e030375dbc260d5af8c4641914b62133dc78f0e5f5"} Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512564 2577 generic.go:356] "Generic (PLEG): container finished" podID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerID="9342028d4930dc413ea83f61915536c0cff49f05a616abba007b93b377153dc5" exitCode=0 Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512716 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerDied","Data":"a7c902e34672bac1b67acf888bdc5ce8f412c6ee5534473736c41ad6e0c374b5"} Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512771 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerDied","Data":"4adc8920e921cf14b994aaf4fddb3e5b214bfc33e2cfce6e9f5cfc35440cab4b"} Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512812 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerDied","Data":"194f4b0a13dcef429305f4a6ab32bf4640a5b0f39d0e868df12a108108f5be0c"} Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerDied","Data":"11bc95a4c081be9ddcf8b453b0181c0e830d9df3d57345febc8ce0ab2ad9edf8"} Sep 15 07:27:39.512942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.512887 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerDied","Data":"9342028d4930dc413ea83f61915536c0cff49f05a616abba007b93b377153dc5"} Sep 15 07:27:39.525745 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.525693 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:39.586063 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586030 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-config-out\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586240 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586071 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-2\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586240 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586106 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-1\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586240 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586127 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-tls-assets\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586240 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586143 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-metrics-client-certs\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586240 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586160 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-config\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586240 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586189 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-kubelet-serving-ca-bundle\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586240 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586232 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-db\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586272 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-serving-certs-ca-bundle\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586296 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-thanos-prometheus-http-client-file\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586328 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-kube-rbac-proxy\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586352 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-trusted-ca-bundle\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586377 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsd57\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-kube-api-access-xsd57\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586408 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-grpc-tls\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586437 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-tls\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586477 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586499 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:39.586629 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586527 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.587123 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586751 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:39.587123 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586820 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:39.587123 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586877 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-web-config\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.587123 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.586937 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-0\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.587123 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.587005 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-metrics-client-ca\") pod \"9b196be3-f862-4158-926d-4c2aa24f4c0b\" (UID: \"9b196be3-f862-4158-926d-4c2aa24f4c0b\") " Sep 15 07:27:39.587659 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.587505 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.587843 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.587826 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.587938 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.587925 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.588090 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.588058 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:27:39.590588 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.590454 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:39.590785 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.590738 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:39.593841 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.593776 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:39.594020 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.593910 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:39.594475 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.594446 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:27:39.595957 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.595817 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw"] Sep 15 07:27:39.596086 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.596052 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:39.596168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.596104 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:27:39.596168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.596141 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-config-out" (OuterVolumeSpecName: "config-out") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:27:39.596662 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.596626 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:39.597418 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.596858 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-config" (OuterVolumeSpecName: "config") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:39.597418 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.596887 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-kube-api-access-xsd57" (OuterVolumeSpecName: "kube-api-access-xsd57") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "kube-api-access-xsd57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:27:39.597418 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.596971 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:39.597930 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.597882 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:39.598102 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:27:39.598000 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd68fd7_3686_466c_a1b1_4f237e1affc1.slice/crio-3cafc1347c004853650fb573722821376444f471d9ea7cdb7e1f608e61d4a668 WatchSource:0}: Error finding container 3cafc1347c004853650fb573722821376444f471d9ea7cdb7e1f608e61d4a668: Status 404 returned error can't find the container with id 3cafc1347c004853650fb573722821376444f471d9ea7cdb7e1f608e61d4a668 Sep 15 07:27:39.598246 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.598222 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:39.598704 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.598673 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:39.608760 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.608737 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-web-config" (OuterVolumeSpecName: "web-config") pod "9b196be3-f862-4158-926d-4c2aa24f4c0b" (UID: "9b196be3-f862-4158-926d-4c2aa24f4c0b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:27:39.689176 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689148 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-metrics-client-ca\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689176 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689171 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-config-out\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689176 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689182 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-tls-assets\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689191 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-metrics-client-certs\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689200 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-config\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689209 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-db\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689220 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689243 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689258 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-kube-rbac-proxy\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689270 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689279 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsd57\" (UniqueName: \"kubernetes.io/projected/9b196be3-f862-4158-926d-4c2aa24f4c0b-kube-api-access-xsd57\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689287 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-grpc-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689294 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689302 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689312 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689320 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b196be3-f862-4158-926d-4c2aa24f4c0b-web-config\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:39.689352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:39.689328 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b196be3-f862-4158-926d-4c2aa24f4c0b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:27:40.517780 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.517681 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" event={"ID":"efd68fd7-3686-466c-a1b1-4f237e1affc1","Type":"ContainerStarted","Data":"3cafc1347c004853650fb573722821376444f471d9ea7cdb7e1f608e61d4a668"} Sep 15 07:27:40.520775 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.520752 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b196be3-f862-4158-926d-4c2aa24f4c0b","Type":"ContainerDied","Data":"cad58398257535e93d9421d9fffe3e396d93a6a4010a0ddee4e61e439b96f4e5"} Sep 15 07:27:40.520892 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.520801 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.520892 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.520817 2577 scope.go:117] "RemoveContainer" containerID="631b045bc27fcdb9d88852e030375dbc260d5af8c4641914b62133dc78f0e5f5" Sep 15 07:27:40.528785 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.528759 2577 scope.go:117] "RemoveContainer" containerID="a7c902e34672bac1b67acf888bdc5ce8f412c6ee5534473736c41ad6e0c374b5" Sep 15 07:27:40.537499 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.537462 2577 scope.go:117] "RemoveContainer" containerID="4adc8920e921cf14b994aaf4fddb3e5b214bfc33e2cfce6e9f5cfc35440cab4b" Sep 15 07:27:40.545355 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.545332 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 07:27:40.545631 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.545616 2577 scope.go:117] "RemoveContainer" containerID="194f4b0a13dcef429305f4a6ab32bf4640a5b0f39d0e868df12a108108f5be0c" Sep 15 07:27:40.552021 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.551996 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 07:27:40.553262 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.553247 2577 scope.go:117] "RemoveContainer" containerID="11bc95a4c081be9ddcf8b453b0181c0e830d9df3d57345febc8ce0ab2ad9edf8" Sep 15 07:27:40.559852 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.559837 2577 scope.go:117] "RemoveContainer" containerID="9342028d4930dc413ea83f61915536c0cff49f05a616abba007b93b377153dc5" Sep 15 07:27:40.566508 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.566473 2577 scope.go:117] "RemoveContainer" containerID="c18499feaf77a6511667fb3ed390e0921adbc60061f22e4fafb4e818b330a14b" Sep 15 07:27:40.579543 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579517 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 07:27:40.579815 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579803 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy-web" Sep 15 07:27:40.579856 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579816 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy-web" Sep 15 07:27:40.579856 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579830 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy-thanos" Sep 15 07:27:40.579856 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579835 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy-thanos" Sep 15 07:27:40.579856 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579843 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="thanos-sidecar" Sep 15 07:27:40.579856 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579849 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="thanos-sidecar" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579860 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579866 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579873 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="init-config-reloader" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579878 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="init-config-reloader" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579895 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="config-reloader" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579900 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="config-reloader" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579907 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="prometheus" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579912 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="prometheus" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579960 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="thanos-sidecar" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579967 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579973 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="config-reloader" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579979 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy-web" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579986 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="prometheus" Sep 15 07:27:40.579999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.579991 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" containerName="kube-rbac-proxy-thanos" Sep 15 07:27:40.583642 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.583624 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.588359 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.588343 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 15 07:27:40.588456 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.588371 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 15 07:27:40.588539 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.588450 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sxp5v\"" Sep 15 07:27:40.588539 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.588472 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 15 07:27:40.588648 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.588539 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 15 07:27:40.588648 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.588477 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 15 07:27:40.589102 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.589056 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 15 07:27:40.591775 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.591755 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 15 07:27:40.591877 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.591859 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 15 07:27:40.592043 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.591984 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 15 07:27:40.592043 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.591759 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 15 07:27:40.592164 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.592128 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 15 07:27:40.592214 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.592171 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-t99qi918frd5\"" Sep 15 07:27:40.592456 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.592440 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 15 07:27:40.594037 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.594016 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 15 07:27:40.595292 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.595276 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 15 07:27:40.601778 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.601755 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 07:27:40.690593 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.690562 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b196be3-f862-4158-926d-4c2aa24f4c0b" path="/var/lib/kubelet/pods/9b196be3-f862-4158-926d-4c2aa24f4c0b/volumes" Sep 15 07:27:40.698784 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.698766 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa9cb324-dd79-4a3e-8f51-563212ff21d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.698862 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.698795 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.698862 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.698816 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.698945 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.698863 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.698945 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.698889 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.698945 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.698937 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-config\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699104 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.698991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699104 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699104 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699251 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699251 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699136 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699251 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f242p\" (UniqueName: \"kubernetes.io/projected/fa9cb324-dd79-4a3e-8f51-563212ff21d3-kube-api-access-f242p\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699251 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699444 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699444 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699284 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699444 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699444 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699372 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699444 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699444 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699421 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.699660 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.699460 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa9cb324-dd79-4a3e-8f51-563212ff21d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.800117 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.800057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.800117 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.800086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.800117 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.800102 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-config\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.800287 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.800172 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.800287 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.800213 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.800287 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.800241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.800432 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.800374 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.800506 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.800428 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.800506 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.800456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f242p\" (UniqueName: \"kubernetes.io/projected/fa9cb324-dd79-4a3e-8f51-563212ff21d3-kube-api-access-f242p\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801092 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801062 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801232 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801203 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801303 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801303 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801298 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801410 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801460 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801460 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801429 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801588 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801454 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801588 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801588 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801537 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa9cb324-dd79-4a3e-8f51-563212ff21d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801588 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa9cb324-dd79-4a3e-8f51-563212ff21d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801776 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.801776 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.801663 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.802887 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.802314 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.803008 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.802939 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.803347 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.803317 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.803347 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.803317 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-config\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.803531 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.803456 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.803701 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.803679 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.803768 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.803731 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.804354 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.804325 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.804440 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.804352 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.804951 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.804882 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.805156 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.805133 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.805551 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.805530 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.805876 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.805857 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.805944 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.805911 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa9cb324-dd79-4a3e-8f51-563212ff21d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.806319 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.806302 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa9cb324-dd79-4a3e-8f51-563212ff21d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.806940 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.806921 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa9cb324-dd79-4a3e-8f51-563212ff21d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.807578 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.807560 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa9cb324-dd79-4a3e-8f51-563212ff21d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.812999 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.812981 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f242p\" (UniqueName: \"kubernetes.io/projected/fa9cb324-dd79-4a3e-8f51-563212ff21d3-kube-api-access-f242p\") pod \"prometheus-k8s-0\" (UID: \"fa9cb324-dd79-4a3e-8f51-563212ff21d3\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:40.894674 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:40.894648 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:27:41.173323 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:41.173297 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 07:27:41.174206 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:27:41.174171 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9cb324_dd79_4a3e_8f51_563212ff21d3.slice/crio-253486c1da714f669485cd64ecb1364f412b3d1868b11c6066c339dff78d8317 WatchSource:0}: Error finding container 253486c1da714f669485cd64ecb1364f412b3d1868b11c6066c339dff78d8317: Status 404 returned error can't find the container with id 253486c1da714f669485cd64ecb1364f412b3d1868b11c6066c339dff78d8317 Sep 15 07:27:41.526926 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:41.526878 2577 generic.go:356] "Generic (PLEG): container finished" podID="fa9cb324-dd79-4a3e-8f51-563212ff21d3" containerID="d508a2a1225e35bcb25943c411785253c32738bf456f853b874da9279403ce24" exitCode=0 Sep 15 07:27:41.527310 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:41.526972 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa9cb324-dd79-4a3e-8f51-563212ff21d3","Type":"ContainerDied","Data":"d508a2a1225e35bcb25943c411785253c32738bf456f853b874da9279403ce24"} Sep 15 07:27:41.527310 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:41.527097 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa9cb324-dd79-4a3e-8f51-563212ff21d3","Type":"ContainerStarted","Data":"253486c1da714f669485cd64ecb1364f412b3d1868b11c6066c339dff78d8317"} Sep 15 07:27:42.535675 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:42.535645 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa9cb324-dd79-4a3e-8f51-563212ff21d3","Type":"ContainerStarted","Data":"fc50471b9e2758f70d5aff7ea8c36aeec9bd12b0cd87641886b7e996e50a1184"} Sep 15 07:27:42.536049 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:42.535706 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa9cb324-dd79-4a3e-8f51-563212ff21d3","Type":"ContainerStarted","Data":"7461c1551c78db154a8a765d99cee142fdaa96ae191fd18526cc9a6e5ce9cda3"} Sep 15 07:27:42.536049 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:42.535732 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa9cb324-dd79-4a3e-8f51-563212ff21d3","Type":"ContainerStarted","Data":"bcb26703ec45e688b0f7ad07219e30cf069feb319cb2304b4e1acf95e7496d88"} Sep 15 07:27:42.536049 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:42.535761 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa9cb324-dd79-4a3e-8f51-563212ff21d3","Type":"ContainerStarted","Data":"a826cf0f859c425043a246771b551c5d9efd067b7edfe01e8c4ebddbfbaad677"} Sep 15 07:27:42.536049 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:42.535784 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa9cb324-dd79-4a3e-8f51-563212ff21d3","Type":"ContainerStarted","Data":"4a471860b2326aeb700281cdbc509bc8b4e14c8c37372df521ea4752c52e5a6c"} Sep 15 07:27:42.536049 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:42.535806 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa9cb324-dd79-4a3e-8f51-563212ff21d3","Type":"ContainerStarted","Data":"b6cc6cbb675a63090a6f5f02e0e0d9155fa96ccc0928f6fb31cb38fd5d8abf17"} Sep 15 07:27:42.573925 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:42.573873 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.556523473 podStartE2EDuration="2.573858233s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.33476ms" isStatefulPod=false podCreationTimestamp="2026-09-15 07:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:27:42.572659512 +0000 UTC m=+154.395130916" watchObservedRunningTime="2026-09-15 07:27:42.573858233 +0000 UTC m=+154.396329679" Sep 15 07:27:44.545789 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:44.545756 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" event={"ID":"efd68fd7-3686-466c-a1b1-4f237e1affc1","Type":"ContainerStarted","Data":"a2bb04658cc5cb0cd9e33675e2a89830888dc48a6c48025b65283f5bf0d099f1"} Sep 15 07:27:44.546158 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:44.545858 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" event={"ID":"efd68fd7-3686-466c-a1b1-4f237e1affc1","Type":"ContainerStarted","Data":"98773b067d6c8ef2720b03e00202c78bc8d1d2b5b5d8048e25b80d645d4320f2"} Sep 15 07:27:44.546158 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:44.545891 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" event={"ID":"efd68fd7-3686-466c-a1b1-4f237e1affc1","Type":"ContainerStarted","Data":"07ad0f8a9e9cfcf1f7abc2d1384195eeb364bf77299fa7034c834eb8453fe3ae"} Sep 15 07:27:44.570399 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:44.570356 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-74b4cf6f8b-69ktw" podStartSLOduration=2.069997705 podStartE2EDuration="6.570342821s" totalImagesPullingTime="4.500345116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:27:44.570105507 +0000 UTC m=+156.392576913" watchObservedRunningTime="2026-09-15 07:27:44.570342821 +0000 UTC m=+156.392814225" Sep 15 07:27:45.895433 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:27:45.895399 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:28:40.895302 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:28:40.895267 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:28:40.910622 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:28:40.910589 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:28:41.787313 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:28:41.787279 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 07:30:08.545758 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:30:08.545728 2577 kubelet.go:1648] "Image garbage collection succeeded" Sep 15 07:32:27.447252 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.447219 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq"] Sep 15 07:32:27.450241 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.450223 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.453520 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.453499 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 15 07:32:27.453750 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.453717 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 15 07:32:27.453847 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.453801 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-pvn7n\"" Sep 15 07:32:27.453847 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.453808 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 15 07:32:27.453847 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.453820 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 15 07:32:27.453847 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.453836 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 15 07:32:27.454080 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.453823 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 15 07:32:27.454136 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.454120 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 15 07:32:27.464146 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.464126 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/659e50d9-c7fc-4d73-b260-994b2380d673-feast-data\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.464250 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.464154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-offline-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.464250 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.464214 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr4mc\" (UniqueName: \"kubernetes.io/projected/659e50d9-c7fc-4d73-b260-994b2380d673-kube-api-access-hr4mc\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.464356 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.464247 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-ui-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.464356 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.464326 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-online-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.464445 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.464385 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-registry-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.467435 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.467415 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq"] Sep 15 07:32:27.565116 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.565086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-offline-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.565242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.565135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr4mc\" (UniqueName: \"kubernetes.io/projected/659e50d9-c7fc-4d73-b260-994b2380d673-kube-api-access-hr4mc\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.565242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.565167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-ui-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.565242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.565185 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-online-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.565242 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.565216 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-registry-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.565397 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.565268 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/659e50d9-c7fc-4d73-b260-994b2380d673-feast-data\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.565662 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.565637 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/659e50d9-c7fc-4d73-b260-994b2380d673-feast-data\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.567831 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.567800 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-online-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.567931 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.567829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-offline-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.567931 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.567851 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-ui-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.567931 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.567829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-registry-tls\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.574660 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.574631 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr4mc\" (UniqueName: \"kubernetes.io/projected/659e50d9-c7fc-4d73-b260-994b2380d673-kube-api-access-hr4mc\") pod \"feast-simple-feast-setup-5c4b96656c-wt8lq\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:27.769448 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:27.769363 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:32:28.044314 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:28.044287 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq"] Sep 15 07:32:28.045389 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:32:28.045363 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod659e50d9_c7fc_4d73_b260_994b2380d673.slice/crio-1d962aa2ce7b4e51db4afed02a2b23fb1a56cce34488c1820d29268e46cfd411 WatchSource:0}: Error finding container 1d962aa2ce7b4e51db4afed02a2b23fb1a56cce34488c1820d29268e46cfd411: Status 404 returned error can't find the container with id 1d962aa2ce7b4e51db4afed02a2b23fb1a56cce34488c1820d29268e46cfd411 Sep 15 07:32:28.047209 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:28.047192 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 07:32:28.669396 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:28.669361 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerStarted","Data":"1d962aa2ce7b4e51db4afed02a2b23fb1a56cce34488c1820d29268e46cfd411"} Sep 15 07:32:54.787930 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:54.787888 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerStarted","Data":"08bca5cf734b97d5086183c8a296d777fbf1fa53e6dc39217422b2818d4f3f11"} Sep 15 07:32:59.812066 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:59.812024 2577 generic.go:356] "Generic (PLEG): container finished" podID="659e50d9-c7fc-4d73-b260-994b2380d673" containerID="08bca5cf734b97d5086183c8a296d777fbf1fa53e6dc39217422b2818d4f3f11" exitCode=0 Sep 15 07:32:59.812564 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:32:59.812100 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerDied","Data":"08bca5cf734b97d5086183c8a296d777fbf1fa53e6dc39217422b2818d4f3f11"} Sep 15 07:33:00.818071 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:00.818036 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerStarted","Data":"bf21c5d0c25942be73beecb2a14e830050e8b6b88725035c71855aeb84165395"} Sep 15 07:33:05.843278 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:05.843189 2577 generic.go:356] "Generic (PLEG): container finished" podID="659e50d9-c7fc-4d73-b260-994b2380d673" containerID="bf21c5d0c25942be73beecb2a14e830050e8b6b88725035c71855aeb84165395" exitCode=0 Sep 15 07:33:05.843278 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:05.843262 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerDied","Data":"bf21c5d0c25942be73beecb2a14e830050e8b6b88725035c71855aeb84165395"} Sep 15 07:33:06.854148 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:06.854105 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerStarted","Data":"4c9e0b412a7b6bd384fdac880581912883d74e9d85a880bb189a34ef42309a09"} Sep 15 07:33:06.854649 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:06.854175 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerStarted","Data":"6ecff4b74b5483790230dc2e0fe069632c4d65b848a8b19c4e98fd7df676118c"} Sep 15 07:33:06.854649 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:06.854199 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerStarted","Data":"6a09f6d25f87bfd9708ad4c374c45098fe55cd39ddafb1386f662c62f0522da7"} Sep 15 07:33:06.854649 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:06.854217 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerStarted","Data":"f5b9bcd994a1b8ac9776dabbf03f5c9953e19ca67b33b12d6759fa434fb47c1a"} Sep 15 07:33:06.882032 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:06.881966 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podStartSLOduration=13.570314746 podStartE2EDuration="39.88194352s" totalImagesPullingTime="26.311628774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:33:06.879037858 +0000 UTC m=+478.701509264" watchObservedRunningTime="2026-09-15 07:33:06.88194352 +0000 UTC m=+478.704414939" Sep 15 07:33:07.770414 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:07.770359 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:07.770647 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:07.770431 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:07.770647 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:07.770454 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:07.770647 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:07.770472 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:09.770377 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:09.770332 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:09.770895 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:09.770403 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:09.770895 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:09.770423 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:09.770895 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:09.770441 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:09.772562 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:09.772519 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused" Sep 15 07:33:09.772781 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:09.772543 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Sep 15 07:33:09.772894 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:09.772546 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 15 07:33:09.772894 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:09.772572 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 15 07:33:12.770525 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:12.770436 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 15 07:33:12.770525 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:12.770445 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused" Sep 15 07:33:12.771071 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:12.770455 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Sep 15 07:33:12.771071 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:12.770458 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 15 07:33:15.770727 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:15.770693 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:15.771191 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:15.770832 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:15.771191 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:15.771093 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:15.771652 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:15.771624 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:15.771989 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:15.771967 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:15.772123 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:15.772106 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:15.776672 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:15.776649 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:33:15.913726 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:33:15.913695 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:34:17.710671 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:17.710576 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq"] Sep 15 07:34:17.711222 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:17.710835 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="registry" containerID="cri-o://f5b9bcd994a1b8ac9776dabbf03f5c9953e19ca67b33b12d6759fa434fb47c1a" gracePeriod=30 Sep 15 07:34:17.711222 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:17.710863 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="online" containerID="cri-o://6a09f6d25f87bfd9708ad4c374c45098fe55cd39ddafb1386f662c62f0522da7" gracePeriod=30 Sep 15 07:34:17.711222 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:17.710864 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="offline" containerID="cri-o://6ecff4b74b5483790230dc2e0fe069632c4d65b848a8b19c4e98fd7df676118c" gracePeriod=30 Sep 15 07:34:17.711222 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:17.710887 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="ui" containerID="cri-o://4c9e0b412a7b6bd384fdac880581912883d74e9d85a880bb189a34ef42309a09" gracePeriod=30 Sep 15 07:34:18.107753 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.107664 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8"] Sep 15 07:34:18.112717 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.112692 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.115643 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.115615 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-6jlq2\"" Sep 15 07:34:18.122596 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.122571 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8"] Sep 15 07:34:18.245355 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.245304 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddkjn\" (UniqueName: \"kubernetes.io/projected/e640a4d3-747c-46d5-a801-d61722482554-kube-api-access-ddkjn\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.245556 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.245418 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-registry-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.245556 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.245458 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e640a4d3-747c-46d5-a801-d61722482554-feast-data\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.245556 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.245508 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-ui-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.245668 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.245604 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-offline-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.245668 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.245625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-online-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.346152 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.346109 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-registry-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.346333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.346172 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e640a4d3-747c-46d5-a801-d61722482554-feast-data\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.346333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.346202 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-ui-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.346333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.346283 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-offline-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.346333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.346310 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-online-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.346560 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.346376 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddkjn\" (UniqueName: \"kubernetes.io/projected/e640a4d3-747c-46d5-a801-d61722482554-kube-api-access-ddkjn\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.347050 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.346990 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e640a4d3-747c-46d5-a801-d61722482554-feast-data\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.349260 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.349231 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-offline-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.349365 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.349259 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-online-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.349699 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.349681 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-registry-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.349767 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.349753 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-ui-tls\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.355886 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.355860 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddkjn\" (UniqueName: \"kubernetes.io/projected/e640a4d3-747c-46d5-a801-d61722482554-kube-api-access-ddkjn\") pod \"feast-simple-feast-setup-5c4b96656c-tlqf8\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.424347 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.424313 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:18.698511 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:18.698467 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8"] Sep 15 07:34:18.701025 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:34:18.700994 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode640a4d3_747c_46d5_a801_d61722482554.slice/crio-8457b0a3e69b199979849e4aa6c7ef00329ab14301d82cc2fe0d7d862de97e24 WatchSource:0}: Error finding container 8457b0a3e69b199979849e4aa6c7ef00329ab14301d82cc2fe0d7d862de97e24: Status 404 returned error can't find the container with id 8457b0a3e69b199979849e4aa6c7ef00329ab14301d82cc2fe0d7d862de97e24 Sep 15 07:34:19.173874 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:19.173792 2577 generic.go:356] "Generic (PLEG): container finished" podID="659e50d9-c7fc-4d73-b260-994b2380d673" containerID="4c9e0b412a7b6bd384fdac880581912883d74e9d85a880bb189a34ef42309a09" exitCode=0 Sep 15 07:34:19.173874 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:19.173819 2577 generic.go:356] "Generic (PLEG): container finished" podID="659e50d9-c7fc-4d73-b260-994b2380d673" containerID="6a09f6d25f87bfd9708ad4c374c45098fe55cd39ddafb1386f662c62f0522da7" exitCode=0 Sep 15 07:34:19.174349 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:19.173870 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerDied","Data":"4c9e0b412a7b6bd384fdac880581912883d74e9d85a880bb189a34ef42309a09"} Sep 15 07:34:19.174349 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:19.173930 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerDied","Data":"6a09f6d25f87bfd9708ad4c374c45098fe55cd39ddafb1386f662c62f0522da7"} Sep 15 07:34:19.175595 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:19.175568 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerStarted","Data":"8451ae4ef7835480b270b1ee642d9e4d049fe20afb9034ae2ba0445252605e97"} Sep 15 07:34:19.175708 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:19.175606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerStarted","Data":"8457b0a3e69b199979849e4aa6c7ef00329ab14301d82cc2fe0d7d862de97e24"} Sep 15 07:34:24.197529 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:24.197473 2577 generic.go:356] "Generic (PLEG): container finished" podID="e640a4d3-747c-46d5-a801-d61722482554" containerID="8451ae4ef7835480b270b1ee642d9e4d049fe20afb9034ae2ba0445252605e97" exitCode=0 Sep 15 07:34:24.197993 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:24.197550 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerDied","Data":"8451ae4ef7835480b270b1ee642d9e4d049fe20afb9034ae2ba0445252605e97"} Sep 15 07:34:25.204168 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:25.204126 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerStarted","Data":"583d6b27fa08c7607a53f7f83fc1fbd201d3172b7c84ff062f85a6162c766b02"} Sep 15 07:34:25.772553 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:25.772503 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 15 07:34:25.910670 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:25.910633 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 15 07:34:30.227637 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:30.227552 2577 generic.go:356] "Generic (PLEG): container finished" podID="e640a4d3-747c-46d5-a801-d61722482554" containerID="583d6b27fa08c7607a53f7f83fc1fbd201d3172b7c84ff062f85a6162c766b02" exitCode=0 Sep 15 07:34:30.228032 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:30.227627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerDied","Data":"583d6b27fa08c7607a53f7f83fc1fbd201d3172b7c84ff062f85a6162c766b02"} Sep 15 07:34:31.238501 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:31.238441 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerStarted","Data":"f84e2a86948a0b67157a7eff2c821574593e79017e85dc5ee1d38859b4304143"} Sep 15 07:34:31.238984 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:31.238528 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerStarted","Data":"49dc6503732d00bea22a230a7e6fc3eb18363e4dc9625bab2ede8f800b98c8a9"} Sep 15 07:34:31.238984 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:31.238551 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerStarted","Data":"707c272905654f62c43cccf4c56ba77e7063056b03877bcea40e345cd3733aa2"} Sep 15 07:34:31.238984 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:31.238569 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerStarted","Data":"1ba2a7e96c62dff97e15bbabf6cbcf558762b59c75cedede44adcf988ecb9f3a"} Sep 15 07:34:31.265451 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:31.265390 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podStartSLOduration=13.265369422 podStartE2EDuration="13.265369422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:34:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:34:31.262927179 +0000 UTC m=+563.085398610" watchObservedRunningTime="2026-09-15 07:34:31.265369422 +0000 UTC m=+563.087840840" Sep 15 07:34:33.425553 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:33.425515 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:33.426106 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:33.426079 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:33.426238 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:33.426116 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:33.426238 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:33.426135 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:33.427981 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:33.427948 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 15 07:34:33.428152 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:33.428127 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 15 07:34:33.428370 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:33.428007 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 15 07:34:33.428474 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:33.427965 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 15 07:34:35.772132 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:35.772078 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 15 07:34:35.911687 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:35.911639 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 15 07:34:36.425243 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:36.425140 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 15 07:34:36.425243 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:36.425139 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 15 07:34:36.425528 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:36.425255 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 15 07:34:36.425528 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:36.425151 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 15 07:34:38.425118 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:38.425073 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:38.425533 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:38.425146 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:38.425533 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:38.425164 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:38.425533 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:38.425181 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:39.425894 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:39.425826 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:39.426528 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:39.426475 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:39.426680 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:39.426667 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:39.426741 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:39.426726 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:39.431252 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:39.431225 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:40.294215 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:40.294185 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:40.294365 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:40.294242 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:40.296892 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:40.296871 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:34:45.772088 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:45.772035 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 15 07:34:45.772765 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:45.772235 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:34:45.911211 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:45.911176 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 15 07:34:45.911367 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:45.911309 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:34:48.330257 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.330221 2577 generic.go:356] "Generic (PLEG): container finished" podID="659e50d9-c7fc-4d73-b260-994b2380d673" containerID="6ecff4b74b5483790230dc2e0fe069632c4d65b848a8b19c4e98fd7df676118c" exitCode=137 Sep 15 07:34:48.330257 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.330256 2577 generic.go:356] "Generic (PLEG): container finished" podID="659e50d9-c7fc-4d73-b260-994b2380d673" containerID="f5b9bcd994a1b8ac9776dabbf03f5c9953e19ca67b33b12d6759fa434fb47c1a" exitCode=137 Sep 15 07:34:48.330703 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.330285 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerDied","Data":"6ecff4b74b5483790230dc2e0fe069632c4d65b848a8b19c4e98fd7df676118c"} Sep 15 07:34:48.330703 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.330329 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerDied","Data":"f5b9bcd994a1b8ac9776dabbf03f5c9953e19ca67b33b12d6759fa434fb47c1a"} Sep 15 07:34:48.360818 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.360795 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:34:48.435725 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.435693 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-online-tls\") pod \"659e50d9-c7fc-4d73-b260-994b2380d673\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " Sep 15 07:34:48.436002 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.435743 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr4mc\" (UniqueName: \"kubernetes.io/projected/659e50d9-c7fc-4d73-b260-994b2380d673-kube-api-access-hr4mc\") pod \"659e50d9-c7fc-4d73-b260-994b2380d673\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " Sep 15 07:34:48.436002 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.435767 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-offline-tls\") pod \"659e50d9-c7fc-4d73-b260-994b2380d673\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " Sep 15 07:34:48.436002 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.435791 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/659e50d9-c7fc-4d73-b260-994b2380d673-feast-data\") pod \"659e50d9-c7fc-4d73-b260-994b2380d673\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " Sep 15 07:34:48.436002 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.435972 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-registry-tls\") pod \"659e50d9-c7fc-4d73-b260-994b2380d673\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " Sep 15 07:34:48.436249 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.436047 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-ui-tls\") pod \"659e50d9-c7fc-4d73-b260-994b2380d673\" (UID: \"659e50d9-c7fc-4d73-b260-994b2380d673\") " Sep 15 07:34:48.436302 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.436247 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/659e50d9-c7fc-4d73-b260-994b2380d673-feast-data" (OuterVolumeSpecName: "feast-data") pod "659e50d9-c7fc-4d73-b260-994b2380d673" (UID: "659e50d9-c7fc-4d73-b260-994b2380d673"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:34:48.436388 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.436370 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/659e50d9-c7fc-4d73-b260-994b2380d673-feast-data\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:34:48.438181 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.438142 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/659e50d9-c7fc-4d73-b260-994b2380d673-kube-api-access-hr4mc" (OuterVolumeSpecName: "kube-api-access-hr4mc") pod "659e50d9-c7fc-4d73-b260-994b2380d673" (UID: "659e50d9-c7fc-4d73-b260-994b2380d673"). InnerVolumeSpecName "kube-api-access-hr4mc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:34:48.438181 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.438150 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-online-tls" (OuterVolumeSpecName: "online-tls") pod "659e50d9-c7fc-4d73-b260-994b2380d673" (UID: "659e50d9-c7fc-4d73-b260-994b2380d673"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:34:48.438336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.438188 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "659e50d9-c7fc-4d73-b260-994b2380d673" (UID: "659e50d9-c7fc-4d73-b260-994b2380d673"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:34:48.438336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.438296 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "659e50d9-c7fc-4d73-b260-994b2380d673" (UID: "659e50d9-c7fc-4d73-b260-994b2380d673"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:34:48.438403 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.438349 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "659e50d9-c7fc-4d73-b260-994b2380d673" (UID: "659e50d9-c7fc-4d73-b260-994b2380d673"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:34:48.537661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.537576 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-registry-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:34:48.537661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.537606 2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-ui-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:34:48.537661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.537633 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-online-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:34:48.537661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.537643 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr4mc\" (UniqueName: \"kubernetes.io/projected/659e50d9-c7fc-4d73-b260-994b2380d673-kube-api-access-hr4mc\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:34:48.537661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:48.537653 2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/659e50d9-c7fc-4d73-b260-994b2380d673-offline-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:34:49.337449 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.337425 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" Sep 15 07:34:49.337856 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.337418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq" event={"ID":"659e50d9-c7fc-4d73-b260-994b2380d673","Type":"ContainerDied","Data":"1d962aa2ce7b4e51db4afed02a2b23fb1a56cce34488c1820d29268e46cfd411"} Sep 15 07:34:49.337856 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.337546 2577 scope.go:117] "RemoveContainer" containerID="4c9e0b412a7b6bd384fdac880581912883d74e9d85a880bb189a34ef42309a09" Sep 15 07:34:49.345789 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.345772 2577 scope.go:117] "RemoveContainer" containerID="6ecff4b74b5483790230dc2e0fe069632c4d65b848a8b19c4e98fd7df676118c" Sep 15 07:34:49.353253 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.353235 2577 scope.go:117] "RemoveContainer" containerID="6a09f6d25f87bfd9708ad4c374c45098fe55cd39ddafb1386f662c62f0522da7" Sep 15 07:34:49.359045 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.359022 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq"] Sep 15 07:34:49.362286 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.362264 2577 scope.go:117] "RemoveContainer" containerID="f5b9bcd994a1b8ac9776dabbf03f5c9953e19ca67b33b12d6759fa434fb47c1a" Sep 15 07:34:49.362383 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.362364 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-wt8lq"] Sep 15 07:34:49.369688 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.369672 2577 scope.go:117] "RemoveContainer" containerID="bf21c5d0c25942be73beecb2a14e830050e8b6b88725035c71855aeb84165395" Sep 15 07:34:49.376600 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:49.376541 2577 scope.go:117] "RemoveContainer" containerID="08bca5cf734b97d5086183c8a296d777fbf1fa53e6dc39217422b2818d4f3f11" Sep 15 07:34:50.689745 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:34:50.689710 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" path="/var/lib/kubelet/pods/659e50d9-c7fc-4d73-b260-994b2380d673/volumes" Sep 15 07:35:43.288797 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.288757 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r"] Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289250 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="offline" Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289269 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="offline" Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289299 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="ui" Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289308 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="ui" Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289327 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="feast-init" Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289337 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="feast-init" Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289357 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="feast-apply" Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289366 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="feast-apply" Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289375 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="online" Sep 15 07:35:43.289382 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289382 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="online" Sep 15 07:35:43.289933 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289396 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="registry" Sep 15 07:35:43.289933 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289403 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="registry" Sep 15 07:35:43.289933 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289502 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="ui" Sep 15 07:35:43.289933 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289515 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="offline" Sep 15 07:35:43.289933 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289530 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="online" Sep 15 07:35:43.289933 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.289542 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="659e50d9-c7fc-4d73-b260-994b2380d673" containerName="registry" Sep 15 07:35:43.292150 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.292130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.295096 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.295062 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 15 07:35:43.295286 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.295270 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 15 07:35:43.295286 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.295281 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 15 07:35:43.295430 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.295305 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 15 07:35:43.295430 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.295353 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 15 07:35:43.295430 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.295313 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-bq46h\"" Sep 15 07:35:43.296352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.296331 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 15 07:35:43.296352 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.296345 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 15 07:35:43.306458 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.306436 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r"] Sep 15 07:35:43.407816 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.407785 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/46ac101e-ef5c-4b42-800e-901227393695-feast-data\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.407965 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.407828 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/46ac101e-ef5c-4b42-800e-901227393695-registry-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.407965 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.407902 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-ui-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.407965 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.407958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhfs9\" (UniqueName: \"kubernetes.io/projected/46ac101e-ef5c-4b42-800e-901227393695-kube-api-access-hhfs9\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.408092 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.407981 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-offline-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.408092 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.408003 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-online-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.508932 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.508901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhfs9\" (UniqueName: \"kubernetes.io/projected/46ac101e-ef5c-4b42-800e-901227393695-kube-api-access-hhfs9\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.509070 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.508935 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-offline-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.509070 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.508963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-online-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.509070 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.509001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/46ac101e-ef5c-4b42-800e-901227393695-feast-data\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.509070 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.509035 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/46ac101e-ef5c-4b42-800e-901227393695-registry-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.509290 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.509080 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-ui-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.509541 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.509510 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/46ac101e-ef5c-4b42-800e-901227393695-feast-data\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.509753 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.509725 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/46ac101e-ef5c-4b42-800e-901227393695-registry-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.511582 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.511551 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-ui-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.511582 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.511569 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-online-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.511697 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.511674 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-offline-tls\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.518375 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.518352 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhfs9\" (UniqueName: \"kubernetes.io/projected/46ac101e-ef5c-4b42-800e-901227393695-kube-api-access-hhfs9\") pod \"feast-simple-feast-remote-setup-79d76d78d5-2zs6r\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.601566 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.601504 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:35:43.875742 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:43.875660 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r"] Sep 15 07:35:43.878056 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:35:43.878033 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46ac101e_ef5c_4b42_800e_901227393695.slice/crio-3db84f354fb4e032c3814efb26ef872b012f3e3142db49f280841d848f97829d WatchSource:0}: Error finding container 3db84f354fb4e032c3814efb26ef872b012f3e3142db49f280841d848f97829d: Status 404 returned error can't find the container with id 3db84f354fb4e032c3814efb26ef872b012f3e3142db49f280841d848f97829d Sep 15 07:35:44.565837 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:44.565789 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerStarted","Data":"2b2d0e6f74750588faac80995bc6a10660e08c5f1fb6a09c7c42253edcd45a39"} Sep 15 07:35:44.566314 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:44.565846 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerStarted","Data":"3db84f354fb4e032c3814efb26ef872b012f3e3142db49f280841d848f97829d"} Sep 15 07:35:49.591757 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:49.591723 2577 generic.go:356] "Generic (PLEG): container finished" podID="46ac101e-ef5c-4b42-800e-901227393695" containerID="2b2d0e6f74750588faac80995bc6a10660e08c5f1fb6a09c7c42253edcd45a39" exitCode=0 Sep 15 07:35:49.592166 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:49.591763 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerDied","Data":"2b2d0e6f74750588faac80995bc6a10660e08c5f1fb6a09c7c42253edcd45a39"} Sep 15 07:35:50.598466 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:35:50.598431 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerStarted","Data":"4f0c63670acdb551d8498451d1020d0316b3e3feca4e74312da3ce73e89e7cc2"} Sep 15 07:36:09.681895 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:09.681858 2577 generic.go:356] "Generic (PLEG): container finished" podID="46ac101e-ef5c-4b42-800e-901227393695" containerID="4f0c63670acdb551d8498451d1020d0316b3e3feca4e74312da3ce73e89e7cc2" exitCode=0 Sep 15 07:36:09.682300 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:09.681932 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerDied","Data":"4f0c63670acdb551d8498451d1020d0316b3e3feca4e74312da3ce73e89e7cc2"} Sep 15 07:36:09.687031 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:09.687013 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 15 07:36:10.691620 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:10.691578 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerStarted","Data":"729947552a0a60987b08feda639f4ac1cb3731fe83a5d123a3c9f254c220b6b7"} Sep 15 07:36:10.692004 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:10.691638 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerStarted","Data":"37fcf62c05f9fb55950f05a1a75f42ea27328e0b9d27a2f677c30e573459e792"} Sep 15 07:36:10.692004 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:10.691662 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerStarted","Data":"65c723401072bcb3dec3f6ceafc2d3c96ac0c8a300bd1973d8bf813191c0d694"} Sep 15 07:36:10.730400 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:10.730339 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podStartSLOduration=27.730320303 podStartE2EDuration="27.730320303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:35:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:36:10.729622529 +0000 UTC m=+662.552093937" watchObservedRunningTime="2026-09-15 07:36:10.730320303 +0000 UTC m=+662.552791708" Sep 15 07:36:13.602098 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:13.602052 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:13.602637 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:13.602118 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:13.602637 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:13.602135 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:13.602637 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:13.602149 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:13.602637 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:13.602163 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:13.602637 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:13.602282 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:13.604100 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:13.604069 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Sep 15 07:36:13.604206 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:13.604126 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 15 07:36:13.604206 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:13.604071 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 15 07:36:16.601985 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:16.601933 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 15 07:36:16.602477 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:16.602029 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 15 07:36:16.602477 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:16.601944 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Sep 15 07:36:19.603064 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:19.603032 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:19.603473 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:19.603346 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:19.604028 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:19.604003 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:19.604212 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:19.604195 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:19.623017 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:19.622995 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:36:19.753800 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:36:19.753770 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:37:21.909167 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:21.909073 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r"] Sep 15 07:37:21.909813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:21.909311 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="online" containerID="cri-o://65c723401072bcb3dec3f6ceafc2d3c96ac0c8a300bd1973d8bf813191c0d694" gracePeriod=30 Sep 15 07:37:21.909813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:21.909370 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="offline" containerID="cri-o://37fcf62c05f9fb55950f05a1a75f42ea27328e0b9d27a2f677c30e573459e792" gracePeriod=30 Sep 15 07:37:21.909813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:21.909408 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="ui" containerID="cri-o://729947552a0a60987b08feda639f4ac1cb3731fe83a5d123a3c9f254c220b6b7" gracePeriod=30 Sep 15 07:37:21.946769 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:21.946740 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8"] Sep 15 07:37:21.947010 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:21.946979 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="registry" containerID="cri-o://1ba2a7e96c62dff97e15bbabf6cbcf558762b59c75cedede44adcf988ecb9f3a" gracePeriod=30 Sep 15 07:37:21.947161 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:21.947008 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="online" containerID="cri-o://707c272905654f62c43cccf4c56ba77e7063056b03877bcea40e345cd3733aa2" gracePeriod=30 Sep 15 07:37:21.947161 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:21.947015 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="ui" containerID="cri-o://f84e2a86948a0b67157a7eff2c821574593e79017e85dc5ee1d38859b4304143" gracePeriod=30 Sep 15 07:37:21.947161 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:21.947018 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="offline" containerID="cri-o://49dc6503732d00bea22a230a7e6fc3eb18363e4dc9625bab2ede8f800b98c8a9" gracePeriod=30 Sep 15 07:37:23.005033 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:23.004998 2577 generic.go:356] "Generic (PLEG): container finished" podID="46ac101e-ef5c-4b42-800e-901227393695" containerID="729947552a0a60987b08feda639f4ac1cb3731fe83a5d123a3c9f254c220b6b7" exitCode=0 Sep 15 07:37:23.005033 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:23.005024 2577 generic.go:356] "Generic (PLEG): container finished" podID="46ac101e-ef5c-4b42-800e-901227393695" containerID="65c723401072bcb3dec3f6ceafc2d3c96ac0c8a300bd1973d8bf813191c0d694" exitCode=0 Sep 15 07:37:23.005463 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:23.005068 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerDied","Data":"729947552a0a60987b08feda639f4ac1cb3731fe83a5d123a3c9f254c220b6b7"} Sep 15 07:37:23.005463 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:23.005110 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerDied","Data":"65c723401072bcb3dec3f6ceafc2d3c96ac0c8a300bd1973d8bf813191c0d694"} Sep 15 07:37:23.007930 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:23.007908 2577 generic.go:356] "Generic (PLEG): container finished" podID="e640a4d3-747c-46d5-a801-d61722482554" containerID="f84e2a86948a0b67157a7eff2c821574593e79017e85dc5ee1d38859b4304143" exitCode=0 Sep 15 07:37:23.007930 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:23.007926 2577 generic.go:356] "Generic (PLEG): container finished" podID="e640a4d3-747c-46d5-a801-d61722482554" containerID="707c272905654f62c43cccf4c56ba77e7063056b03877bcea40e345cd3733aa2" exitCode=0 Sep 15 07:37:23.008086 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:23.007960 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerDied","Data":"f84e2a86948a0b67157a7eff2c821574593e79017e85dc5ee1d38859b4304143"} Sep 15 07:37:23.008086 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:23.007983 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerDied","Data":"707c272905654f62c43cccf4c56ba77e7063056b03877bcea40e345cd3733aa2"} Sep 15 07:37:29.603855 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:29.603809 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 15 07:37:29.737201 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:29.737155 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 15 07:37:30.294702 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:30.294651 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 15 07:37:30.294900 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:30.294661 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 15 07:37:39.603942 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:39.603888 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 15 07:37:39.736252 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:39.736214 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 15 07:37:40.293918 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:40.293866 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 15 07:37:40.294109 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:40.293884 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 15 07:37:49.603906 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:49.603859 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 15 07:37:49.604330 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:49.603988 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:37:49.737069 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:49.737029 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 15 07:37:49.737228 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:49.737139 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:37:50.294545 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:50.294475 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 15 07:37:50.294744 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:50.294512 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 15 07:37:50.294744 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:50.294637 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:37:50.294744 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:50.294704 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:37:52.135430 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.135399 2577 generic.go:356] "Generic (PLEG): container finished" podID="e640a4d3-747c-46d5-a801-d61722482554" containerID="49dc6503732d00bea22a230a7e6fc3eb18363e4dc9625bab2ede8f800b98c8a9" exitCode=137 Sep 15 07:37:52.135430 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.135426 2577 generic.go:356] "Generic (PLEG): container finished" podID="e640a4d3-747c-46d5-a801-d61722482554" containerID="1ba2a7e96c62dff97e15bbabf6cbcf558762b59c75cedede44adcf988ecb9f3a" exitCode=137 Sep 15 07:37:52.135866 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.135464 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerDied","Data":"49dc6503732d00bea22a230a7e6fc3eb18363e4dc9625bab2ede8f800b98c8a9"} Sep 15 07:37:52.135866 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.135522 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerDied","Data":"1ba2a7e96c62dff97e15bbabf6cbcf558762b59c75cedede44adcf988ecb9f3a"} Sep 15 07:37:52.138173 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.138152 2577 generic.go:356] "Generic (PLEG): container finished" podID="46ac101e-ef5c-4b42-800e-901227393695" containerID="37fcf62c05f9fb55950f05a1a75f42ea27328e0b9d27a2f677c30e573459e792" exitCode=137 Sep 15 07:37:52.138282 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.138224 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerDied","Data":"37fcf62c05f9fb55950f05a1a75f42ea27328e0b9d27a2f677c30e573459e792"} Sep 15 07:37:52.562863 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.562840 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:37:52.600965 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.600933 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:37:52.622652 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.622627 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddkjn\" (UniqueName: \"kubernetes.io/projected/e640a4d3-747c-46d5-a801-d61722482554-kube-api-access-ddkjn\") pod \"e640a4d3-747c-46d5-a801-d61722482554\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " Sep 15 07:37:52.622749 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.622660 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-offline-tls\") pod \"e640a4d3-747c-46d5-a801-d61722482554\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " Sep 15 07:37:52.622749 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.622676 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-online-tls\") pod \"e640a4d3-747c-46d5-a801-d61722482554\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " Sep 15 07:37:52.622749 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.622709 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/46ac101e-ef5c-4b42-800e-901227393695-registry-tls\") pod \"46ac101e-ef5c-4b42-800e-901227393695\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " Sep 15 07:37:52.622749 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.622740 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-offline-tls\") pod \"46ac101e-ef5c-4b42-800e-901227393695\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " Sep 15 07:37:52.622889 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.622759 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-ui-tls\") pod \"e640a4d3-747c-46d5-a801-d61722482554\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " Sep 15 07:37:52.622889 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.622777 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/46ac101e-ef5c-4b42-800e-901227393695-feast-data\") pod \"46ac101e-ef5c-4b42-800e-901227393695\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " Sep 15 07:37:52.622889 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.622799 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e640a4d3-747c-46d5-a801-d61722482554-feast-data\") pod \"e640a4d3-747c-46d5-a801-d61722482554\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " Sep 15 07:37:52.623689 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.623277 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46ac101e-ef5c-4b42-800e-901227393695-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "46ac101e-ef5c-4b42-800e-901227393695" (UID: "46ac101e-ef5c-4b42-800e-901227393695"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:37:52.623808 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.623711 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46ac101e-ef5c-4b42-800e-901227393695-feast-data" (OuterVolumeSpecName: "feast-data") pod "46ac101e-ef5c-4b42-800e-901227393695" (UID: "46ac101e-ef5c-4b42-800e-901227393695"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:37:52.625351 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.623998 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e640a4d3-747c-46d5-a801-d61722482554-feast-data" (OuterVolumeSpecName: "feast-data") pod "e640a4d3-747c-46d5-a801-d61722482554" (UID: "e640a4d3-747c-46d5-a801-d61722482554"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:37:52.625462 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.625383 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhfs9\" (UniqueName: \"kubernetes.io/projected/46ac101e-ef5c-4b42-800e-901227393695-kube-api-access-hhfs9\") pod \"46ac101e-ef5c-4b42-800e-901227393695\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " Sep 15 07:37:52.625462 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.625432 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-ui-tls\") pod \"46ac101e-ef5c-4b42-800e-901227393695\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " Sep 15 07:37:52.626169 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.625803 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-registry-tls\") pod \"e640a4d3-747c-46d5-a801-d61722482554\" (UID: \"e640a4d3-747c-46d5-a801-d61722482554\") " Sep 15 07:37:52.626169 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.625836 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-online-tls\") pod \"46ac101e-ef5c-4b42-800e-901227393695\" (UID: \"46ac101e-ef5c-4b42-800e-901227393695\") " Sep 15 07:37:52.626169 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.626110 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/46ac101e-ef5c-4b42-800e-901227393695-registry-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.626169 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.626126 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/46ac101e-ef5c-4b42-800e-901227393695-feast-data\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.626169 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.626138 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e640a4d3-747c-46d5-a801-d61722482554-feast-data\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.626451 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.626373 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e640a4d3-747c-46d5-a801-d61722482554" (UID: "e640a4d3-747c-46d5-a801-d61722482554"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:37:52.626528 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.626467 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-online-tls" (OuterVolumeSpecName: "online-tls") pod "e640a4d3-747c-46d5-a801-d61722482554" (UID: "e640a4d3-747c-46d5-a801-d61722482554"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:37:52.626644 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.626599 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e640a4d3-747c-46d5-a801-d61722482554" (UID: "e640a4d3-747c-46d5-a801-d61722482554"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:37:52.626965 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.626920 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "46ac101e-ef5c-4b42-800e-901227393695" (UID: "46ac101e-ef5c-4b42-800e-901227393695"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:37:52.626965 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.626932 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46ac101e-ef5c-4b42-800e-901227393695-kube-api-access-hhfs9" (OuterVolumeSpecName: "kube-api-access-hhfs9") pod "46ac101e-ef5c-4b42-800e-901227393695" (UID: "46ac101e-ef5c-4b42-800e-901227393695"). InnerVolumeSpecName "kube-api-access-hhfs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:37:52.627263 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.627226 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e640a4d3-747c-46d5-a801-d61722482554-kube-api-access-ddkjn" (OuterVolumeSpecName: "kube-api-access-ddkjn") pod "e640a4d3-747c-46d5-a801-d61722482554" (UID: "e640a4d3-747c-46d5-a801-d61722482554"). InnerVolumeSpecName "kube-api-access-ddkjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:37:52.628101 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.628077 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "46ac101e-ef5c-4b42-800e-901227393695" (UID: "46ac101e-ef5c-4b42-800e-901227393695"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:37:52.628256 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.628238 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e640a4d3-747c-46d5-a801-d61722482554" (UID: "e640a4d3-747c-46d5-a801-d61722482554"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:37:52.628760 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.628734 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-online-tls" (OuterVolumeSpecName: "online-tls") pod "46ac101e-ef5c-4b42-800e-901227393695" (UID: "46ac101e-ef5c-4b42-800e-901227393695"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:37:52.726785 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.726758 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-registry-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.726888 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.726804 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-online-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.726888 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.726814 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddkjn\" (UniqueName: \"kubernetes.io/projected/e640a4d3-747c-46d5-a801-d61722482554-kube-api-access-ddkjn\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.726888 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.726824 2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-offline-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.726888 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.726834 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-online-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.726888 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.726846 2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-offline-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.726888 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.726854 2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e640a4d3-747c-46d5-a801-d61722482554-ui-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.726888 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.726865 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhfs9\" (UniqueName: \"kubernetes.io/projected/46ac101e-ef5c-4b42-800e-901227393695-kube-api-access-hhfs9\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:52.726888 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:52.726883 2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/46ac101e-ef5c-4b42-800e-901227393695-ui-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:37:53.145167 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.145084 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" Sep 15 07:37:53.145167 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.145119 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8" event={"ID":"e640a4d3-747c-46d5-a801-d61722482554","Type":"ContainerDied","Data":"8457b0a3e69b199979849e4aa6c7ef00329ab14301d82cc2fe0d7d862de97e24"} Sep 15 07:37:53.145714 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.145189 2577 scope.go:117] "RemoveContainer" containerID="f84e2a86948a0b67157a7eff2c821574593e79017e85dc5ee1d38859b4304143" Sep 15 07:37:53.148403 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.148371 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" event={"ID":"46ac101e-ef5c-4b42-800e-901227393695","Type":"ContainerDied","Data":"3db84f354fb4e032c3814efb26ef872b012f3e3142db49f280841d848f97829d"} Sep 15 07:37:53.148518 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.148389 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r" Sep 15 07:37:53.154405 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.154386 2577 scope.go:117] "RemoveContainer" containerID="49dc6503732d00bea22a230a7e6fc3eb18363e4dc9625bab2ede8f800b98c8a9" Sep 15 07:37:53.161674 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.161658 2577 scope.go:117] "RemoveContainer" containerID="707c272905654f62c43cccf4c56ba77e7063056b03877bcea40e345cd3733aa2" Sep 15 07:37:53.165631 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.165610 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r"] Sep 15 07:37:53.169351 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.169331 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-79d76d78d5-2zs6r"] Sep 15 07:37:53.169548 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.169527 2577 scope.go:117] "RemoveContainer" containerID="1ba2a7e96c62dff97e15bbabf6cbcf558762b59c75cedede44adcf988ecb9f3a" Sep 15 07:37:53.176727 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.176697 2577 scope.go:117] "RemoveContainer" containerID="583d6b27fa08c7607a53f7f83fc1fbd201d3172b7c84ff062f85a6162c766b02" Sep 15 07:37:53.181812 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.181793 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8"] Sep 15 07:37:53.184278 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.184259 2577 scope.go:117] "RemoveContainer" containerID="8451ae4ef7835480b270b1ee642d9e4d049fe20afb9034ae2ba0445252605e97" Sep 15 07:37:53.185393 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.185372 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5c4b96656c-tlqf8"] Sep 15 07:37:53.191467 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.191448 2577 scope.go:117] "RemoveContainer" containerID="729947552a0a60987b08feda639f4ac1cb3731fe83a5d123a3c9f254c220b6b7" Sep 15 07:37:53.198223 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.198208 2577 scope.go:117] "RemoveContainer" containerID="37fcf62c05f9fb55950f05a1a75f42ea27328e0b9d27a2f677c30e573459e792" Sep 15 07:37:53.205328 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.205310 2577 scope.go:117] "RemoveContainer" containerID="65c723401072bcb3dec3f6ceafc2d3c96ac0c8a300bd1973d8bf813191c0d694" Sep 15 07:37:53.212243 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.212214 2577 scope.go:117] "RemoveContainer" containerID="4f0c63670acdb551d8498451d1020d0316b3e3feca4e74312da3ce73e89e7cc2" Sep 15 07:37:53.219333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:53.219317 2577 scope.go:117] "RemoveContainer" containerID="2b2d0e6f74750588faac80995bc6a10660e08c5f1fb6a09c7c42253edcd45a39" Sep 15 07:37:54.690468 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:54.690438 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46ac101e-ef5c-4b42-800e-901227393695" path="/var/lib/kubelet/pods/46ac101e-ef5c-4b42-800e-901227393695/volumes" Sep 15 07:37:54.690997 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:37:54.690983 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e640a4d3-747c-46d5-a801-d61722482554" path="/var/lib/kubelet/pods/e640a4d3-747c-46d5-a801-d61722482554/volumes" Sep 15 07:38:04.678610 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.678573 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-49qfz"] Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.678935 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="online" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.678960 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="online" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.678979 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="online" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.678987 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="online" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.678996 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="ui" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679001 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="ui" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679009 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="registry" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679014 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="registry" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679021 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="feast-apply" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679026 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="feast-apply" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679034 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="offline" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679040 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="offline" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679056 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="feast-init" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679074 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="feast-init" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679092 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="ui" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679100 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="ui" Sep 15 07:38:04.679099 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679110 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="feast-init" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679116 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="feast-init" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679125 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="feast-apply" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679130 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="feast-apply" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679138 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="offline" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679143 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="offline" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679201 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="offline" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679210 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="online" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679218 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="registry" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679255 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="offline" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679266 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="ui" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679277 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e640a4d3-747c-46d5-a801-d61722482554" containerName="ui" Sep 15 07:38:04.679986 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.679293 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="46ac101e-ef5c-4b42-800e-901227393695" containerName="online" Sep 15 07:38:04.682185 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.682162 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-49qfz" Sep 15 07:38:04.685335 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.685316 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-j64rw\"" Sep 15 07:38:04.685609 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.685593 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 15 07:38:04.686725 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.686696 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 15 07:38:04.692345 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.692326 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-49qfz"] Sep 15 07:38:04.723852 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.723820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftsc5\" (UniqueName: \"kubernetes.io/projected/86a8bea2-0692-47eb-8393-7586dea86aa4-kube-api-access-ftsc5\") pod \"redis-69bbd798b6-49qfz\" (UID: \"86a8bea2-0692-47eb-8393-7586dea86aa4\") " pod="test-ns-feast/redis-69bbd798b6-49qfz" Sep 15 07:38:04.825080 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.825031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftsc5\" (UniqueName: \"kubernetes.io/projected/86a8bea2-0692-47eb-8393-7586dea86aa4-kube-api-access-ftsc5\") pod \"redis-69bbd798b6-49qfz\" (UID: \"86a8bea2-0692-47eb-8393-7586dea86aa4\") " pod="test-ns-feast/redis-69bbd798b6-49qfz" Sep 15 07:38:04.834208 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.834180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftsc5\" (UniqueName: \"kubernetes.io/projected/86a8bea2-0692-47eb-8393-7586dea86aa4-kube-api-access-ftsc5\") pod \"redis-69bbd798b6-49qfz\" (UID: \"86a8bea2-0692-47eb-8393-7586dea86aa4\") " pod="test-ns-feast/redis-69bbd798b6-49qfz" Sep 15 07:38:04.998657 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:04.998624 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-49qfz" Sep 15 07:38:05.267144 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:05.267118 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-49qfz"] Sep 15 07:38:05.269654 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:38:05.269620 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86a8bea2_0692_47eb_8393_7586dea86aa4.slice/crio-39ef47f8cb09b2d0cc4ad657ad39f17da8b78e8afe3ba471926f67102c89821d WatchSource:0}: Error finding container 39ef47f8cb09b2d0cc4ad657ad39f17da8b78e8afe3ba471926f67102c89821d: Status 404 returned error can't find the container with id 39ef47f8cb09b2d0cc4ad657ad39f17da8b78e8afe3ba471926f67102c89821d Sep 15 07:38:05.271526 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:05.271509 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 07:38:06.205306 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:06.205268 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-49qfz" event={"ID":"86a8bea2-0692-47eb-8393-7586dea86aa4","Type":"ContainerStarted","Data":"39ef47f8cb09b2d0cc4ad657ad39f17da8b78e8afe3ba471926f67102c89821d"} Sep 15 07:38:07.210941 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:07.210911 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-49qfz" event={"ID":"86a8bea2-0692-47eb-8393-7586dea86aa4","Type":"ContainerStarted","Data":"6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb"} Sep 15 07:38:07.226679 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:07.226625 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-49qfz" podStartSLOduration=1.75170621 podStartE2EDuration="3.226606079s" totalImagesPullingTime="1.474899869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:38:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:38:07.225501615 +0000 UTC m=+779.047973016" watchObservedRunningTime="2026-09-15 07:38:07.226606079 +0000 UTC m=+779.049077483" Sep 15 07:38:13.480243 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.480203 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t"] Sep 15 07:38:13.485836 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.485813 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.488865 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.488842 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 15 07:38:13.488974 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.488940 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 15 07:38:13.489160 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.489147 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 15 07:38:13.489438 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.489424 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-bmnvj\"" Sep 15 07:38:13.490257 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.490241 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 15 07:38:13.490306 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.490252 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 15 07:38:13.495813 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.495792 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t"] Sep 15 07:38:13.602705 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.602671 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-registry-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.602705 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.602717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-offline-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.602928 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.602811 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-feast-data\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.602928 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.602861 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-online-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.602928 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.602879 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kd9r\" (UniqueName: \"kubernetes.io/projected/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-kube-api-access-5kd9r\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.703764 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.703732 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-registry-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.703939 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.703772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-offline-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.703939 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.703806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-feast-data\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.703939 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.703836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-online-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.703939 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.703851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kd9r\" (UniqueName: \"kubernetes.io/projected/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-kube-api-access-5kd9r\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.704160 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:38:13.703968 2577 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Sep 15 07:38:13.704160 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:38:13.704040 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-online-tls podName:979fa59f-c3ee-4d74-9df2-a4ff7d08be15 nodeName:}" failed. No retries permitted until 2026-09-15 07:38:14.204019631 +0000 UTC m=+786.026491018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-online-tls") pod "feast-credit-scoring-848cc64d9b-ppz4t" (UID: "979fa59f-c3ee-4d74-9df2-a4ff7d08be15") : secret "feast-credit-scoring-online-tls" not found Sep 15 07:38:13.704305 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.704285 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-feast-data\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.706201 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.706182 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-offline-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.706261 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.706198 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-registry-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:13.715178 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:13.715153 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kd9r\" (UniqueName: \"kubernetes.io/projected/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-kube-api-access-5kd9r\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:14.208451 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:14.208391 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-online-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:14.210861 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:14.210831 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-online-tls\") pod \"feast-credit-scoring-848cc64d9b-ppz4t\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:14.396387 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:14.396331 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:14.674724 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:14.674697 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t"] Sep 15 07:38:14.676462 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:38:14.676426 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979fa59f_c3ee_4d74_9df2_a4ff7d08be15.slice/crio-3fca3036758ade68058104eedd2a9d35bc1fdec8ff9227887e7155577bd78ebb WatchSource:0}: Error finding container 3fca3036758ade68058104eedd2a9d35bc1fdec8ff9227887e7155577bd78ebb: Status 404 returned error can't find the container with id 3fca3036758ade68058104eedd2a9d35bc1fdec8ff9227887e7155577bd78ebb Sep 15 07:38:15.247697 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:15.247658 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerStarted","Data":"696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe"} Sep 15 07:38:15.247903 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:15.247738 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerStarted","Data":"3fca3036758ade68058104eedd2a9d35bc1fdec8ff9227887e7155577bd78ebb"} Sep 15 07:38:16.253533 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:16.253472 2577 generic.go:356] "Generic (PLEG): container finished" podID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerID="696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe" exitCode=0 Sep 15 07:38:16.253916 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:16.253511 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerDied","Data":"696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe"} Sep 15 07:38:17.261046 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:17.261010 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerStarted","Data":"099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225"} Sep 15 07:38:23.288388 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:23.288352 2577 generic.go:356] "Generic (PLEG): container finished" podID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerID="099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225" exitCode=0 Sep 15 07:38:23.288759 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:23.288419 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerDied","Data":"099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225"} Sep 15 07:38:24.297980 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:24.297934 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerStarted","Data":"852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d"} Sep 15 07:38:24.298466 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:24.298019 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerStarted","Data":"c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e"} Sep 15 07:38:24.298466 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:24.298053 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerStarted","Data":"41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5"} Sep 15 07:38:24.323576 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:24.323518 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podStartSLOduration=11.32349629 podStartE2EDuration="11.32349629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:38:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:38:24.320502281 +0000 UTC m=+796.142973698" watchObservedRunningTime="2026-09-15 07:38:24.32349629 +0000 UTC m=+796.145967689" Sep 15 07:38:24.396452 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:24.396412 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:24.396642 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:24.396539 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:24.396642 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:24.396578 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:26.397089 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:26.397048 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:26.397594 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:26.397164 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:26.397594 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:26.397198 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:26.398667 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:26.398635 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 15 07:38:26.398783 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:26.398666 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 15 07:38:26.398783 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:26.398689 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:38:29.397628 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:29.397582 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 15 07:38:29.398210 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:29.397590 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 15 07:38:29.398448 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:29.397755 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:38:32.396903 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:32.396799 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 15 07:38:32.397726 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:32.397700 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:32.398606 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:32.398582 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:32.403351 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:32.403326 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:33.352286 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:33.352256 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:35.397355 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:35.397311 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 15 07:38:38.398080 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:38.398049 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:38:38.398891 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:38:38.398870 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:39:41.876652 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:41.876614 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t"] Sep 15 07:39:41.877127 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:41.876875 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="registry" containerID="cri-o://41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5" gracePeriod=30 Sep 15 07:39:41.877127 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:41.876938 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="offline" containerID="cri-o://852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d" gracePeriod=30 Sep 15 07:39:41.877127 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:41.876938 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="online" containerID="cri-o://c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e" gracePeriod=30 Sep 15 07:39:41.895998 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:41.895972 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-49qfz"] Sep 15 07:39:41.896145 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:41.896120 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-49qfz" podUID="86a8bea2-0692-47eb-8393-7586dea86aa4" containerName="redis" containerID="cri-o://6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb" gracePeriod=30 Sep 15 07:39:42.127858 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.127801 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-49qfz" Sep 15 07:39:42.179955 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.179930 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftsc5\" (UniqueName: \"kubernetes.io/projected/86a8bea2-0692-47eb-8393-7586dea86aa4-kube-api-access-ftsc5\") pod \"86a8bea2-0692-47eb-8393-7586dea86aa4\" (UID: \"86a8bea2-0692-47eb-8393-7586dea86aa4\") " Sep 15 07:39:42.181950 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.181912 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86a8bea2-0692-47eb-8393-7586dea86aa4-kube-api-access-ftsc5" (OuterVolumeSpecName: "kube-api-access-ftsc5") pod "86a8bea2-0692-47eb-8393-7586dea86aa4" (UID: "86a8bea2-0692-47eb-8393-7586dea86aa4"). InnerVolumeSpecName "kube-api-access-ftsc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:39:42.281396 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.281352 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftsc5\" (UniqueName: \"kubernetes.io/projected/86a8bea2-0692-47eb-8393-7586dea86aa4-kube-api-access-ftsc5\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:39:42.649659 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.649626 2577 generic.go:356] "Generic (PLEG): container finished" podID="86a8bea2-0692-47eb-8393-7586dea86aa4" containerID="6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb" exitCode=0 Sep 15 07:39:42.649828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.649690 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-49qfz" Sep 15 07:39:42.649828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.649704 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-49qfz" event={"ID":"86a8bea2-0692-47eb-8393-7586dea86aa4","Type":"ContainerDied","Data":"6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb"} Sep 15 07:39:42.649828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.649748 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-49qfz" event={"ID":"86a8bea2-0692-47eb-8393-7586dea86aa4","Type":"ContainerDied","Data":"39ef47f8cb09b2d0cc4ad657ad39f17da8b78e8afe3ba471926f67102c89821d"} Sep 15 07:39:42.649828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.649766 2577 scope.go:117] "RemoveContainer" containerID="6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb" Sep 15 07:39:42.665332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.665313 2577 scope.go:117] "RemoveContainer" containerID="6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb" Sep 15 07:39:42.665628 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:39:42.665609 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb\": container with ID starting with 6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb not found: ID does not exist" containerID="6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb" Sep 15 07:39:42.665684 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.665637 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb"} err="failed to get container status \"6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb\": rpc error: code = NotFound desc = could not find container \"6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb\": container with ID starting with 6b61b9f7120e2828f6085b158b7b043ed9dc8dea4d8617e6ef57d239e95407cb not found: ID does not exist" Sep 15 07:39:42.677375 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.677351 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-49qfz"] Sep 15 07:39:42.679373 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.679353 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-49qfz"] Sep 15 07:39:42.689282 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:42.689265 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86a8bea2-0692-47eb-8393-7586dea86aa4" path="/var/lib/kubelet/pods/86a8bea2-0692-47eb-8393-7586dea86aa4/volumes" Sep 15 07:39:43.349797 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:43.349754 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:39:43.656080 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:43.656001 2577 generic.go:356] "Generic (PLEG): container finished" podID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerID="c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e" exitCode=0 Sep 15 07:39:43.656223 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:43.656075 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerDied","Data":"c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e"} Sep 15 07:39:53.349838 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:39:53.349792 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:40:03.350238 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:03.350151 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:40:03.350661 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:03.350286 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:40:12.399113 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.399063 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 15 07:40:12.521256 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.521233 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:40:12.535231 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.535210 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-online-tls\") pod \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " Sep 15 07:40:12.535347 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.535304 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-offline-tls\") pod \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " Sep 15 07:40:12.535408 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.535356 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-feast-data\") pod \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " Sep 15 07:40:12.535408 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.535400 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kd9r\" (UniqueName: \"kubernetes.io/projected/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-kube-api-access-5kd9r\") pod \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " Sep 15 07:40:12.537288 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.537262 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "979fa59f-c3ee-4d74-9df2-a4ff7d08be15" (UID: "979fa59f-c3ee-4d74-9df2-a4ff7d08be15"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:40:12.537379 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.537320 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-online-tls" (OuterVolumeSpecName: "online-tls") pod "979fa59f-c3ee-4d74-9df2-a4ff7d08be15" (UID: "979fa59f-c3ee-4d74-9df2-a4ff7d08be15"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:40:12.537446 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.537415 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-kube-api-access-5kd9r" (OuterVolumeSpecName: "kube-api-access-5kd9r") pod "979fa59f-c3ee-4d74-9df2-a4ff7d08be15" (UID: "979fa59f-c3ee-4d74-9df2-a4ff7d08be15"). InnerVolumeSpecName "kube-api-access-5kd9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:40:12.543558 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.543533 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-feast-data" (OuterVolumeSpecName: "feast-data") pod "979fa59f-c3ee-4d74-9df2-a4ff7d08be15" (UID: "979fa59f-c3ee-4d74-9df2-a4ff7d08be15"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:40:12.636009 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.635983 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-registry-tls\") pod \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\" (UID: \"979fa59f-c3ee-4d74-9df2-a4ff7d08be15\") " Sep 15 07:40:12.636177 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.636160 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kd9r\" (UniqueName: \"kubernetes.io/projected/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-kube-api-access-5kd9r\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:40:12.636222 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.636184 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-online-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:40:12.636222 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.636197 2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-offline-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:40:12.636222 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.636209 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-feast-data\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:40:12.637900 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.637882 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "979fa59f-c3ee-4d74-9df2-a4ff7d08be15" (UID: "979fa59f-c3ee-4d74-9df2-a4ff7d08be15"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:40:12.736642 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.736616 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/979fa59f-c3ee-4d74-9df2-a4ff7d08be15-registry-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:40:12.776590 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.776563 2577 generic.go:356] "Generic (PLEG): container finished" podID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerID="852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d" exitCode=137 Sep 15 07:40:12.776590 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.776587 2577 generic.go:356] "Generic (PLEG): container finished" podID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerID="41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5" exitCode=137 Sep 15 07:40:12.776756 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.776630 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerDied","Data":"852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d"} Sep 15 07:40:12.776756 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.776632 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" Sep 15 07:40:12.776756 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.776673 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerDied","Data":"41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5"} Sep 15 07:40:12.776756 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.776692 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t" event={"ID":"979fa59f-c3ee-4d74-9df2-a4ff7d08be15","Type":"ContainerDied","Data":"3fca3036758ade68058104eedd2a9d35bc1fdec8ff9227887e7155577bd78ebb"} Sep 15 07:40:12.776756 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.776715 2577 scope.go:117] "RemoveContainer" containerID="852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d" Sep 15 07:40:12.785145 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.785124 2577 scope.go:117] "RemoveContainer" containerID="c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e" Sep 15 07:40:12.792760 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.792744 2577 scope.go:117] "RemoveContainer" containerID="41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5" Sep 15 07:40:12.796021 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.795998 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t"] Sep 15 07:40:12.800438 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.800415 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-848cc64d9b-ppz4t"] Sep 15 07:40:12.801815 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.801798 2577 scope.go:117] "RemoveContainer" containerID="099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225" Sep 15 07:40:12.808846 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.808830 2577 scope.go:117] "RemoveContainer" containerID="696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe" Sep 15 07:40:12.815867 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.815849 2577 scope.go:117] "RemoveContainer" containerID="852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d" Sep 15 07:40:12.816118 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:40:12.816099 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d\": container with ID starting with 852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d not found: ID does not exist" containerID="852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d" Sep 15 07:40:12.816177 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.816126 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d"} err="failed to get container status \"852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d\": rpc error: code = NotFound desc = could not find container \"852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d\": container with ID starting with 852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d not found: ID does not exist" Sep 15 07:40:12.816177 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.816150 2577 scope.go:117] "RemoveContainer" containerID="c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e" Sep 15 07:40:12.816408 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:40:12.816390 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e\": container with ID starting with c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e not found: ID does not exist" containerID="c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e" Sep 15 07:40:12.816453 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.816417 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e"} err="failed to get container status \"c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e\": rpc error: code = NotFound desc = could not find container \"c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e\": container with ID starting with c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e not found: ID does not exist" Sep 15 07:40:12.816453 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.816434 2577 scope.go:117] "RemoveContainer" containerID="41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5" Sep 15 07:40:12.816681 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:40:12.816663 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5\": container with ID starting with 41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5 not found: ID does not exist" containerID="41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5" Sep 15 07:40:12.816722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.816688 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5"} err="failed to get container status \"41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5\": rpc error: code = NotFound desc = could not find container \"41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5\": container with ID starting with 41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5 not found: ID does not exist" Sep 15 07:40:12.816722 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.816701 2577 scope.go:117] "RemoveContainer" containerID="099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225" Sep 15 07:40:12.816933 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:40:12.816917 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225\": container with ID starting with 099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225 not found: ID does not exist" containerID="099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225" Sep 15 07:40:12.816984 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.816936 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225"} err="failed to get container status \"099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225\": rpc error: code = NotFound desc = could not find container \"099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225\": container with ID starting with 099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225 not found: ID does not exist" Sep 15 07:40:12.816984 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.816947 2577 scope.go:117] "RemoveContainer" containerID="696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe" Sep 15 07:40:12.817229 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:40:12.817173 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe\": container with ID starting with 696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe not found: ID does not exist" containerID="696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe" Sep 15 07:40:12.817293 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.817237 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe"} err="failed to get container status \"696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe\": rpc error: code = NotFound desc = could not find container \"696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe\": container with ID starting with 696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe not found: ID does not exist" Sep 15 07:40:12.817293 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.817257 2577 scope.go:117] "RemoveContainer" containerID="852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d" Sep 15 07:40:12.817584 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.817556 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d"} err="failed to get container status \"852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d\": rpc error: code = NotFound desc = could not find container \"852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d\": container with ID starting with 852ccc3de33d2e0d1cd2b9c2e65e8eb102c576f0f093005c9f779738801aee1d not found: ID does not exist" Sep 15 07:40:12.817584 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.817577 2577 scope.go:117] "RemoveContainer" containerID="c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e" Sep 15 07:40:12.817772 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.817752 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e"} err="failed to get container status \"c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e\": rpc error: code = NotFound desc = could not find container \"c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e\": container with ID starting with c01ccd1c9e306dd824bc370daf515302e4aff5755588f08f748291b2ab01d62e not found: ID does not exist" Sep 15 07:40:12.817817 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.817776 2577 scope.go:117] "RemoveContainer" containerID="41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5" Sep 15 07:40:12.818032 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.818012 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5"} err="failed to get container status \"41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5\": rpc error: code = NotFound desc = could not find container \"41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5\": container with ID starting with 41e18dea91751f770fac2f888eba1514216663562e18285a5d84804e1a54fee5 not found: ID does not exist" Sep 15 07:40:12.818110 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.818034 2577 scope.go:117] "RemoveContainer" containerID="099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225" Sep 15 07:40:12.818252 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.818236 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225"} err="failed to get container status \"099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225\": rpc error: code = NotFound desc = could not find container \"099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225\": container with ID starting with 099df5d54940ac513dbb98944f1c9424d399d2ed3fd918ce1ddb343140bbc225 not found: ID does not exist" Sep 15 07:40:12.818320 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.818254 2577 scope.go:117] "RemoveContainer" containerID="696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe" Sep 15 07:40:12.818455 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:12.818439 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe"} err="failed to get container status \"696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe\": rpc error: code = NotFound desc = could not find container \"696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe\": container with ID starting with 696f34d8a8bca74f64c02fd4db51ab8d6e0754b5fb0ed32b9c3502fec61381fe not found: ID does not exist" Sep 15 07:40:14.690425 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:40:14.690393 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" path="/var/lib/kubelet/pods/979fa59f-c3ee-4d74-9df2-a4ff7d08be15/volumes" Sep 15 07:46:25.779143 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779067 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5"] Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779389 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86a8bea2-0692-47eb-8393-7586dea86aa4" containerName="redis" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779400 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="86a8bea2-0692-47eb-8393-7586dea86aa4" containerName="redis" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779413 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="registry" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779418 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="registry" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779428 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="online" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779434 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="online" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779440 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="offline" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779445 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="offline" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779453 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="feast-apply" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779458 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="feast-apply" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779466 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="feast-init" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779472 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="feast-init" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779541 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="online" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779550 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="86a8bea2-0692-47eb-8393-7586dea86aa4" containerName="redis" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779557 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="registry" Sep 15 07:46:25.781332 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.779565 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="979fa59f-c3ee-4d74-9df2-a4ff7d08be15" containerName="offline" Sep 15 07:46:25.782681 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.782666 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.785712 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.785690 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-zr6ks\"" Sep 15 07:46:25.785828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.785715 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 15 07:46:25.785828 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.785765 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 15 07:46:25.785949 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.785911 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 15 07:46:25.785949 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.785917 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 15 07:46:25.785949 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.785922 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 15 07:46:25.786110 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.786014 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 15 07:46:25.794531 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.794512 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5"] Sep 15 07:46:25.826971 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.826950 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-online-tls\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.827096 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.826990 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-registry-tls\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.827096 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.827067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnhss\" (UniqueName: \"kubernetes.io/projected/bcbea111-92b2-4c94-b4db-e20f1e52dd69-kube-api-access-pnhss\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.827176 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.827147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bcbea111-92b2-4c94-b4db-e20f1e52dd69-feast-data\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.928273 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.928244 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bcbea111-92b2-4c94-b4db-e20f1e52dd69-feast-data\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.928421 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.928301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-online-tls\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.928421 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.928323 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-registry-tls\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.928421 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.928347 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnhss\" (UniqueName: \"kubernetes.io/projected/bcbea111-92b2-4c94-b4db-e20f1e52dd69-kube-api-access-pnhss\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.928613 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:46:25.928464 2577 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 15 07:46:25.928613 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:46:25.928552 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-registry-tls podName:bcbea111-92b2-4c94-b4db-e20f1e52dd69 nodeName:}" failed. No retries permitted until 2026-09-15 07:46:26.428536876 +0000 UTC m=+1278.251008263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-registry-tls") pod "feast-credit-scoring-76574c4bb9-d5cb5" (UID: "bcbea111-92b2-4c94-b4db-e20f1e52dd69") : secret "feast-credit-scoring-registry-tls" not found Sep 15 07:46:25.928613 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.928600 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bcbea111-92b2-4c94-b4db-e20f1e52dd69-feast-data\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.930557 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.930535 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-online-tls\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:25.940093 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:25.940073 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnhss\" (UniqueName: \"kubernetes.io/projected/bcbea111-92b2-4c94-b4db-e20f1e52dd69-kube-api-access-pnhss\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:26.432700 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:26.432661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-registry-tls\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:26.435011 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:26.434981 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-registry-tls\") pod \"feast-credit-scoring-76574c4bb9-d5cb5\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:26.692006 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:26.691920 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:26.964743 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:26.964718 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5"] Sep 15 07:46:26.965689 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:46:26.965665 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbea111_92b2_4c94_b4db_e20f1e52dd69.slice/crio-419a8e370374bf4cc701244122ea94bde8a8e24db6a498824fac31c432c9cb0c WatchSource:0}: Error finding container 419a8e370374bf4cc701244122ea94bde8a8e24db6a498824fac31c432c9cb0c: Status 404 returned error can't find the container with id 419a8e370374bf4cc701244122ea94bde8a8e24db6a498824fac31c432c9cb0c Sep 15 07:46:27.270709 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:27.270619 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerStarted","Data":"87deaaba5c9c1834721e3ab538a46dade8ceb5ca3986036d74352ffe8035e440"} Sep 15 07:46:27.270709 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:27.270685 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerStarted","Data":"419a8e370374bf4cc701244122ea94bde8a8e24db6a498824fac31c432c9cb0c"} Sep 15 07:46:28.276392 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:28.276310 2577 generic.go:356] "Generic (PLEG): container finished" podID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerID="87deaaba5c9c1834721e3ab538a46dade8ceb5ca3986036d74352ffe8035e440" exitCode=0 Sep 15 07:46:28.276770 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:28.276398 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerDied","Data":"87deaaba5c9c1834721e3ab538a46dade8ceb5ca3986036d74352ffe8035e440"} Sep 15 07:46:29.282262 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:29.282224 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerStarted","Data":"7c1368a09254e2a67c15f7608c5f6e61e57419b91cb8e89553e02a37baa3c0cb"} Sep 15 07:46:36.314842 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:36.314810 2577 generic.go:356] "Generic (PLEG): container finished" podID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerID="7c1368a09254e2a67c15f7608c5f6e61e57419b91cb8e89553e02a37baa3c0cb" exitCode=0 Sep 15 07:46:36.315222 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:36.314885 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerDied","Data":"7c1368a09254e2a67c15f7608c5f6e61e57419b91cb8e89553e02a37baa3c0cb"} Sep 15 07:46:37.322596 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:37.322563 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerStarted","Data":"f58955f45bdbfe7c9700429b5e328515052477ed8b5afd2332d6976e75290372"} Sep 15 07:46:37.322973 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:37.322613 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerStarted","Data":"98142543699edd890a37d4e7212bfd3712321fb4828cb2de7824fb99ee259e42"} Sep 15 07:46:37.348155 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:37.348094 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podStartSLOduration=12.348070573 podStartE2EDuration="12.348070573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:46:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:46:37.344688575 +0000 UTC m=+1289.167160028" watchObservedRunningTime="2026-09-15 07:46:37.348070573 +0000 UTC m=+1289.170541979" Sep 15 07:46:38.692375 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:38.692344 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:38.692876 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:38.692393 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:38.693937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:38.693908 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 15 07:46:38.693937 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:38.693923 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:46:41.692539 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:41.692495 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 15 07:46:41.692929 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:41.692493 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:46:44.692336 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:44.692283 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 15 07:46:44.698037 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:44.698008 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:44.698396 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:44.698373 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:44.701312 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:44.701291 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:46.692945 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:46.692910 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:47.692950 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:47.692906 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 15 07:46:50.692718 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:50.692683 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:50.693404 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:50.693384 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:46:51.651387 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.651351 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq"] Sep 15 07:46:51.655449 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.655427 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.662383 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.658422 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-9vkj9\"" Sep 15 07:46:51.662383 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.658772 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 15 07:46:51.662383 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.658974 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 15 07:46:51.662383 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.659183 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 15 07:46:51.665809 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.665786 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq"] Sep 15 07:46:51.764681 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.764657 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-online-tls\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.765046 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.764715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-feast-data\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.765046 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.764792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-registry-tls\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.765046 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.764838 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79z76\" (UniqueName: \"kubernetes.io/projected/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-kube-api-access-79z76\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.865886 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.865857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-online-tls\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.866045 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.865914 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-feast-data\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.866045 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:46:51.866008 2577 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Sep 15 07:46:51.866154 ip-10-0-135-38 kubenswrapper[2577]: E0915 07:46:51.866071 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-online-tls podName:6f226aff-4121-4e15-a0ec-ce9cbe8b5099 nodeName:}" failed. No retries permitted until 2026-09-15 07:46:52.3660541 +0000 UTC m=+1304.188525506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-online-tls") pod "feast-driver-ranking-5b94b7bb7d-9vhnq" (UID: "6f226aff-4121-4e15-a0ec-ce9cbe8b5099") : secret "feast-driver-ranking-online-tls" not found Sep 15 07:46:51.866154 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.866089 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-registry-tls\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.866154 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.866130 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79z76\" (UniqueName: \"kubernetes.io/projected/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-kube-api-access-79z76\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.866312 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.866252 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-feast-data\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.868601 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.868577 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-registry-tls\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:51.874247 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:51.874225 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79z76\" (UniqueName: \"kubernetes.io/projected/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-kube-api-access-79z76\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:52.370083 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:52.370042 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-online-tls\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:52.372366 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:52.372341 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-online-tls\") pod \"feast-driver-ranking-5b94b7bb7d-9vhnq\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:52.570930 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:52.570894 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:46:52.845024 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:52.844984 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq"] Sep 15 07:46:52.845360 ip-10-0-135-38 kubenswrapper[2577]: W0915 07:46:52.845333 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f226aff_4121_4e15_a0ec_ce9cbe8b5099.slice/crio-aa26f19711ed6b6ca3f176f91da6e98893a42cd4d9c72e7cf5b79cff533064b1 WatchSource:0}: Error finding container aa26f19711ed6b6ca3f176f91da6e98893a42cd4d9c72e7cf5b79cff533064b1: Status 404 returned error can't find the container with id aa26f19711ed6b6ca3f176f91da6e98893a42cd4d9c72e7cf5b79cff533064b1 Sep 15 07:46:53.395619 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:53.395576 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerStarted","Data":"519d0f723641f054df808759cb169ff23fcb89505ea363b81f800e18cb83ed6c"} Sep 15 07:46:53.395812 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:53.395647 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerStarted","Data":"aa26f19711ed6b6ca3f176f91da6e98893a42cd4d9c72e7cf5b79cff533064b1"} Sep 15 07:46:58.417115 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:58.417075 2577 generic.go:356] "Generic (PLEG): container finished" podID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerID="519d0f723641f054df808759cb169ff23fcb89505ea363b81f800e18cb83ed6c" exitCode=0 Sep 15 07:46:58.417556 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:58.417156 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerDied","Data":"519d0f723641f054df808759cb169ff23fcb89505ea363b81f800e18cb83ed6c"} Sep 15 07:46:59.422948 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:46:59.422915 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerStarted","Data":"cfa71237513e47799d113030919e8e4bbaa23b824b8944b357c4ae9aca3e2deb"} Sep 15 07:47:06.455436 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:06.455404 2577 generic.go:356] "Generic (PLEG): container finished" podID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerID="cfa71237513e47799d113030919e8e4bbaa23b824b8944b357c4ae9aca3e2deb" exitCode=0 Sep 15 07:47:06.455855 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:06.455495 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerDied","Data":"cfa71237513e47799d113030919e8e4bbaa23b824b8944b357c4ae9aca3e2deb"} Sep 15 07:47:07.462911 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:07.462875 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerStarted","Data":"05b82070b6937c2fadd154481e850f6decffb95b92d2c76d9c269e82d45b1303"} Sep 15 07:47:07.463297 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:07.462936 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerStarted","Data":"caac0d1c2889aab8ab49f98953791e11c32a05c32a00a55d6e0d3872baf42e33"} Sep 15 07:47:07.487562 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:07.487511 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podStartSLOduration=16.487474382 podStartE2EDuration="16.487474382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:46:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:47:07.486690582 +0000 UTC m=+1319.309162013" watchObservedRunningTime="2026-09-15 07:47:07.487474382 +0000 UTC m=+1319.309945786" Sep 15 07:47:07.572057 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:07.572006 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:47:07.572267 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:07.572089 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:47:07.573567 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:07.573528 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 15 07:47:07.573688 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:07.573566 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 15 07:47:10.571313 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:10.571267 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 15 07:47:10.571744 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:10.571257 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 15 07:47:12.571977 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:12.571938 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:47:12.572607 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:12.572591 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:47:13.572283 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:13.572229 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 15 07:47:13.578323 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:13.578294 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:47:14.504208 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:14.504172 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:47:16.571814 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:16.571752 2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 15 07:47:19.572371 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:19.572339 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:47:19.573167 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:47:19.573145 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:48:40.768981 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:40.768874 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5"] Sep 15 07:48:40.770245 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:40.770206 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="registry" containerID="cri-o://98142543699edd890a37d4e7212bfd3712321fb4828cb2de7824fb99ee259e42" gracePeriod=30 Sep 15 07:48:40.770245 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:40.770206 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="online" containerID="cri-o://f58955f45bdbfe7c9700429b5e328515052477ed8b5afd2332d6976e75290372" gracePeriod=30 Sep 15 07:48:40.772069 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:40.772045 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq"] Sep 15 07:48:40.772281 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:40.772241 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="registry" containerID="cri-o://caac0d1c2889aab8ab49f98953791e11c32a05c32a00a55d6e0d3872baf42e33" gracePeriod=30 Sep 15 07:48:40.772364 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:40.772307 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="online" containerID="cri-o://05b82070b6937c2fadd154481e850f6decffb95b92d2c76d9c269e82d45b1303" gracePeriod=30 Sep 15 07:48:41.876820 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:41.876727 2577 generic.go:356] "Generic (PLEG): container finished" podID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerID="05b82070b6937c2fadd154481e850f6decffb95b92d2c76d9c269e82d45b1303" exitCode=0 Sep 15 07:48:41.876820 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:41.876798 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerDied","Data":"05b82070b6937c2fadd154481e850f6decffb95b92d2c76d9c269e82d45b1303"} Sep 15 07:48:41.878963 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:41.878940 2577 generic.go:356] "Generic (PLEG): container finished" podID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerID="f58955f45bdbfe7c9700429b5e328515052477ed8b5afd2332d6976e75290372" exitCode=0 Sep 15 07:48:41.879097 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:41.879008 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerDied","Data":"f58955f45bdbfe7c9700429b5e328515052477ed8b5afd2332d6976e75290372"} Sep 15 07:48:44.500972 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:44.500926 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 15 07:48:44.699244 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:44.699205 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:48:54.501449 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:54.501404 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 15 07:48:54.699552 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:48:54.699511 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:49:04.501873 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:04.501783 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 15 07:49:04.502273 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:04.501912 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:49:04.698582 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:04.698544 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:49:04.698760 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:04.698675 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:49:10.994994 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:10.994958 2577 generic.go:356] "Generic (PLEG): container finished" podID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerID="caac0d1c2889aab8ab49f98953791e11c32a05c32a00a55d6e0d3872baf42e33" exitCode=137 Sep 15 07:49:10.995361 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:10.995026 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerDied","Data":"caac0d1c2889aab8ab49f98953791e11c32a05c32a00a55d6e0d3872baf42e33"} Sep 15 07:49:10.997326 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:10.997305 2577 generic.go:356] "Generic (PLEG): container finished" podID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerID="98142543699edd890a37d4e7212bfd3712321fb4828cb2de7824fb99ee259e42" exitCode=137 Sep 15 07:49:10.997451 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:10.997377 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerDied","Data":"98142543699edd890a37d4e7212bfd3712321fb4828cb2de7824fb99ee259e42"} Sep 15 07:49:11.422576 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.422555 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:49:11.425435 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.425416 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:49:11.552509 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.552423 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-online-tls\") pod \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " Sep 15 07:49:11.552509 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.552455 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnhss\" (UniqueName: \"kubernetes.io/projected/bcbea111-92b2-4c94-b4db-e20f1e52dd69-kube-api-access-pnhss\") pod \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " Sep 15 07:49:11.552509 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.552500 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bcbea111-92b2-4c94-b4db-e20f1e52dd69-feast-data\") pod \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " Sep 15 07:49:11.552761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.552529 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-registry-tls\") pod \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " Sep 15 07:49:11.552761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.552552 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79z76\" (UniqueName: \"kubernetes.io/projected/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-kube-api-access-79z76\") pod \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " Sep 15 07:49:11.552761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.552579 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-online-tls\") pod \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " Sep 15 07:49:11.552761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.552631 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-feast-data\") pod \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\" (UID: \"6f226aff-4121-4e15-a0ec-ce9cbe8b5099\") " Sep 15 07:49:11.552761 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.552673 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-registry-tls\") pod \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\" (UID: \"bcbea111-92b2-4c94-b4db-e20f1e52dd69\") " Sep 15 07:49:11.553346 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.553244 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-feast-data" (OuterVolumeSpecName: "feast-data") pod "6f226aff-4121-4e15-a0ec-ce9cbe8b5099" (UID: "6f226aff-4121-4e15-a0ec-ce9cbe8b5099"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:49:11.555200 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.555157 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcbea111-92b2-4c94-b4db-e20f1e52dd69-kube-api-access-pnhss" (OuterVolumeSpecName: "kube-api-access-pnhss") pod "bcbea111-92b2-4c94-b4db-e20f1e52dd69" (UID: "bcbea111-92b2-4c94-b4db-e20f1e52dd69"). InnerVolumeSpecName "kube-api-access-pnhss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:49:11.555321 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.555208 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-online-tls" (OuterVolumeSpecName: "online-tls") pod "bcbea111-92b2-4c94-b4db-e20f1e52dd69" (UID: "bcbea111-92b2-4c94-b4db-e20f1e52dd69"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:49:11.555321 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.555245 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-kube-api-access-79z76" (OuterVolumeSpecName: "kube-api-access-79z76") pod "6f226aff-4121-4e15-a0ec-ce9cbe8b5099" (UID: "6f226aff-4121-4e15-a0ec-ce9cbe8b5099"). InnerVolumeSpecName "kube-api-access-79z76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:49:11.555321 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.555245 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-online-tls" (OuterVolumeSpecName: "online-tls") pod "6f226aff-4121-4e15-a0ec-ce9cbe8b5099" (UID: "6f226aff-4121-4e15-a0ec-ce9cbe8b5099"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:49:11.555321 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.555263 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6f226aff-4121-4e15-a0ec-ce9cbe8b5099" (UID: "6f226aff-4121-4e15-a0ec-ce9cbe8b5099"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:49:11.555321 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.555298 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bcbea111-92b2-4c94-b4db-e20f1e52dd69" (UID: "bcbea111-92b2-4c94-b4db-e20f1e52dd69"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:49:11.562580 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.562560 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcbea111-92b2-4c94-b4db-e20f1e52dd69-feast-data" (OuterVolumeSpecName: "feast-data") pod "bcbea111-92b2-4c94-b4db-e20f1e52dd69" (UID: "bcbea111-92b2-4c94-b4db-e20f1e52dd69"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:49:11.653854 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.653829 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-feast-data\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:49:11.653854 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.653849 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-registry-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:49:11.653854 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.653858 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-online-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:49:11.654095 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.653867 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnhss\" (UniqueName: \"kubernetes.io/projected/bcbea111-92b2-4c94-b4db-e20f1e52dd69-kube-api-access-pnhss\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:49:11.654095 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.653876 2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bcbea111-92b2-4c94-b4db-e20f1e52dd69-feast-data\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:49:11.654095 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.653884 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-registry-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:49:11.654095 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.653891 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79z76\" (UniqueName: \"kubernetes.io/projected/6f226aff-4121-4e15-a0ec-ce9cbe8b5099-kube-api-access-79z76\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:49:11.654095 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:11.653900 2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bcbea111-92b2-4c94-b4db-e20f1e52dd69-online-tls\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 07:49:12.004248 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.004207 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" event={"ID":"6f226aff-4121-4e15-a0ec-ce9cbe8b5099","Type":"ContainerDied","Data":"aa26f19711ed6b6ca3f176f91da6e98893a42cd4d9c72e7cf5b79cff533064b1"} Sep 15 07:49:12.004721 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.004223 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq" Sep 15 07:49:12.004721 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.004312 2577 scope.go:117] "RemoveContainer" containerID="05b82070b6937c2fadd154481e850f6decffb95b92d2c76d9c269e82d45b1303" Sep 15 07:49:12.007020 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.006995 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" event={"ID":"bcbea111-92b2-4c94-b4db-e20f1e52dd69","Type":"ContainerDied","Data":"419a8e370374bf4cc701244122ea94bde8a8e24db6a498824fac31c432c9cb0c"} Sep 15 07:49:12.007170 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.007048 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5" Sep 15 07:49:12.014716 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.014527 2577 scope.go:117] "RemoveContainer" containerID="caac0d1c2889aab8ab49f98953791e11c32a05c32a00a55d6e0d3872baf42e33" Sep 15 07:49:12.024636 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.024616 2577 scope.go:117] "RemoveContainer" containerID="cfa71237513e47799d113030919e8e4bbaa23b824b8944b357c4ae9aca3e2deb" Sep 15 07:49:12.030333 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.030311 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq"] Sep 15 07:49:12.033250 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.033229 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5b94b7bb7d-9vhnq"] Sep 15 07:49:12.036445 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.036428 2577 scope.go:117] "RemoveContainer" containerID="519d0f723641f054df808759cb169ff23fcb89505ea363b81f800e18cb83ed6c" Sep 15 07:49:12.044316 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.044270 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5"] Sep 15 07:49:12.044365 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.044348 2577 scope.go:117] "RemoveContainer" containerID="f58955f45bdbfe7c9700429b5e328515052477ed8b5afd2332d6976e75290372" Sep 15 07:49:12.048096 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.048075 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-76574c4bb9-d5cb5"] Sep 15 07:49:12.052760 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.052745 2577 scope.go:117] "RemoveContainer" containerID="98142543699edd890a37d4e7212bfd3712321fb4828cb2de7824fb99ee259e42" Sep 15 07:49:12.060840 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.060819 2577 scope.go:117] "RemoveContainer" containerID="7c1368a09254e2a67c15f7608c5f6e61e57419b91cb8e89553e02a37baa3c0cb" Sep 15 07:49:12.067756 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.067738 2577 scope.go:117] "RemoveContainer" containerID="87deaaba5c9c1834721e3ab538a46dade8ceb5ca3986036d74352ffe8035e440" Sep 15 07:49:12.689549 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.689514 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" path="/var/lib/kubelet/pods/6f226aff-4121-4e15-a0ec-ce9cbe8b5099/volumes" Sep 15 07:49:12.690026 ip-10-0-135-38 kubenswrapper[2577]: I0915 07:49:12.690012 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" path="/var/lib/kubelet/pods/bcbea111-92b2-4c94-b4db-e20f1e52dd69/volumes" Sep 15 08:00:22.918923 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.918879 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j7lv4/must-gather-z528s"] Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919364 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="feast-apply" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919384 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="feast-apply" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919395 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="registry" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919404 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="registry" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919420 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="online" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919428 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="online" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919440 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="feast-init" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919449 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="feast-init" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919477 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="registry" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919502 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="registry" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919514 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="feast-init" Sep 15 08:00:22.919526 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919523 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="feast-init" Sep 15 08:00:22.920135 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919534 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="feast-apply" Sep 15 08:00:22.920135 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919543 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="feast-apply" Sep 15 08:00:22.920135 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919553 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="online" Sep 15 08:00:22.920135 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919560 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="online" Sep 15 08:00:22.920135 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919645 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="online" Sep 15 08:00:22.920135 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919661 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="registry" Sep 15 08:00:22.920135 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919674 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbea111-92b2-4c94-b4db-e20f1e52dd69" containerName="registry" Sep 15 08:00:22.920135 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.919684 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f226aff-4121-4e15-a0ec-ce9cbe8b5099" containerName="online" Sep 15 08:00:22.923462 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.923438 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:22.926211 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.926190 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j7lv4\"/\"openshift-service-ca.crt\"" Sep 15 08:00:22.927779 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.927756 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j7lv4\"/\"kube-root-ca.crt\"" Sep 15 08:00:22.927999 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.927982 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j7lv4\"/\"default-dockercfg-4r95w\"" Sep 15 08:00:22.930127 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:22.930100 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j7lv4/must-gather-z528s"] Sep 15 08:00:23.080812 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.080773 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4brcm\" (UniqueName: \"kubernetes.io/projected/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-kube-api-access-4brcm\") pod \"must-gather-z528s\" (UID: \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\") " pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:23.080975 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.080906 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-must-gather-output\") pod \"must-gather-z528s\" (UID: \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\") " pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:23.181633 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.181596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-must-gather-output\") pod \"must-gather-z528s\" (UID: \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\") " pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:23.181796 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.181648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4brcm\" (UniqueName: \"kubernetes.io/projected/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-kube-api-access-4brcm\") pod \"must-gather-z528s\" (UID: \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\") " pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:23.181961 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.181923 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-must-gather-output\") pod \"must-gather-z528s\" (UID: \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\") " pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:23.190417 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.190384 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4brcm\" (UniqueName: \"kubernetes.io/projected/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-kube-api-access-4brcm\") pod \"must-gather-z528s\" (UID: \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\") " pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:23.246104 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.246069 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:23.515177 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.515094 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j7lv4/must-gather-z528s"] Sep 15 08:00:23.518137 ip-10-0-135-38 kubenswrapper[2577]: W0915 08:00:23.518109 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod496a83f0_ee0a_4217_9f01_44cc90ff2f6c.slice/crio-a98fd387430e541750c1196630dc4093d576126c645f1b9469dfa587ef4ff088 WatchSource:0}: Error finding container a98fd387430e541750c1196630dc4093d576126c645f1b9469dfa587ef4ff088: Status 404 returned error can't find the container with id a98fd387430e541750c1196630dc4093d576126c645f1b9469dfa587ef4ff088 Sep 15 08:00:23.520006 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.519988 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 08:00:23.665621 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:23.665587 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j7lv4/must-gather-z528s" event={"ID":"496a83f0-ee0a-4217-9f01-44cc90ff2f6c","Type":"ContainerStarted","Data":"a98fd387430e541750c1196630dc4093d576126c645f1b9469dfa587ef4ff088"} Sep 15 08:00:32.708701 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:32.708615 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j7lv4/must-gather-z528s" event={"ID":"496a83f0-ee0a-4217-9f01-44cc90ff2f6c","Type":"ContainerStarted","Data":"312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0"} Sep 15 08:00:32.708701 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:32.708660 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j7lv4/must-gather-z528s" event={"ID":"496a83f0-ee0a-4217-9f01-44cc90ff2f6c","Type":"ContainerStarted","Data":"f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746"} Sep 15 08:00:32.726051 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:32.726001 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j7lv4/must-gather-z528s" podStartSLOduration=2.063298903 podStartE2EDuration="10.725987404s" totalImagesPullingTime="8.662688501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 08:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 08:00:32.725418437 +0000 UTC m=+2124.547889842" watchObservedRunningTime="2026-09-15 08:00:32.725987404 +0000 UTC m=+2124.548458827" Sep 15 08:00:41.751091 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:41.751062 2577 generic.go:356] "Generic (PLEG): container finished" podID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" containerID="f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746" exitCode=0 Sep 15 08:00:41.751535 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:41.751120 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j7lv4/must-gather-z528s" event={"ID":"496a83f0-ee0a-4217-9f01-44cc90ff2f6c","Type":"ContainerDied","Data":"f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746"} Sep 15 08:00:41.751535 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:41.751418 2577 scope.go:117] "RemoveContainer" containerID="f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746" Sep 15 08:00:42.724888 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:42.724856 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j7lv4_must-gather-z528s_496a83f0-ee0a-4217-9f01-44cc90ff2f6c/gather/0.log" Sep 15 08:00:46.047908 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:46.047873 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-v8tt9_e623486b-2e21-4cfc-844d-25fefc0f4418/global-pull-secret-syncer/0.log" Sep 15 08:00:46.229370 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:46.229329 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l6k5h_90846c11-f046-48a5-a8a0-de0e57db04ee/konnectivity-agent/0.log" Sep 15 08:00:46.296470 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:46.296445 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-38.ec2.internal_1bc2623acdeb0a77dc30a9def2462fc4/haproxy/0.log" Sep 15 08:00:48.042348 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.042302 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-j7lv4/must-gather-z528s"] Sep 15 08:00:48.042781 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.042503 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-j7lv4/must-gather-z528s" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" containerName="copy" containerID="cri-o://312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0" gracePeriod=2 Sep 15 08:00:48.049242 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.049214 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-j7lv4/must-gather-z528s"] Sep 15 08:00:48.050192 ip-10-0-135-38 kubenswrapper[2577]: E0915 08:00:48.050165 2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-j7lv4\"/\"must-gather-z528s\": pods \"must-gather-z528s\" not found" pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:48.271613 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.271592 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j7lv4_must-gather-z528s_496a83f0-ee0a-4217-9f01-44cc90ff2f6c/copy/0.log" Sep 15 08:00:48.273226 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.273209 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:48.275941 ip-10-0-135-38 kubenswrapper[2577]: E0915 08:00:48.275918 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-z528s\" is forbidden: User \"system:node:ip-10-0-135-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j7lv4\": no relationship found between node 'ip-10-0-135-38.ec2.internal' and this object" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:48.406701 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.406629 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4brcm\" (UniqueName: \"kubernetes.io/projected/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-kube-api-access-4brcm\") pod \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\" (UID: \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\") " Sep 15 08:00:48.406701 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.406677 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-must-gather-output\") pod \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\" (UID: \"496a83f0-ee0a-4217-9f01-44cc90ff2f6c\") " Sep 15 08:00:48.407058 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.407034 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "496a83f0-ee0a-4217-9f01-44cc90ff2f6c" (UID: "496a83f0-ee0a-4217-9f01-44cc90ff2f6c"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 08:00:48.408818 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.408786 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-kube-api-access-4brcm" (OuterVolumeSpecName: "kube-api-access-4brcm") pod "496a83f0-ee0a-4217-9f01-44cc90ff2f6c" (UID: "496a83f0-ee0a-4217-9f01-44cc90ff2f6c"). InnerVolumeSpecName "kube-api-access-4brcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 08:00:48.507154 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.507125 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4brcm\" (UniqueName: \"kubernetes.io/projected/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-kube-api-access-4brcm\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 08:00:48.507154 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.507149 2577 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/496a83f0-ee0a-4217-9f01-44cc90ff2f6c-must-gather-output\") on node \"ip-10-0-135-38.ec2.internal\" DevicePath \"\"" Sep 15 08:00:48.693988 ip-10-0-135-38 kubenswrapper[2577]: E0915 08:00:48.693953 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-z528s\" is forbidden: User \"system:node:ip-10-0-135-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j7lv4\": no relationship found between node 'ip-10-0-135-38.ec2.internal' and this object" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:48.695747 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.695723 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" path="/var/lib/kubelet/pods/496a83f0-ee0a-4217-9f01-44cc90ff2f6c/volumes" Sep 15 08:00:48.781557 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.781529 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j7lv4_must-gather-z528s_496a83f0-ee0a-4217-9f01-44cc90ff2f6c/copy/0.log" Sep 15 08:00:48.781892 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.781871 2577 generic.go:356] "Generic (PLEG): container finished" podID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" containerID="312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0" exitCode=143 Sep 15 08:00:48.781945 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.781931 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j7lv4/must-gather-z528s" Sep 15 08:00:48.781982 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.781952 2577 scope.go:117] "RemoveContainer" containerID="312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0" Sep 15 08:00:48.789476 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.789447 2577 scope.go:117] "RemoveContainer" containerID="f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746" Sep 15 08:00:48.801233 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.801217 2577 scope.go:117] "RemoveContainer" containerID="312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0" Sep 15 08:00:48.801515 ip-10-0-135-38 kubenswrapper[2577]: E0915 08:00:48.801496 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0\": container with ID starting with 312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0 not found: ID does not exist" containerID="312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0" Sep 15 08:00:48.801560 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.801522 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0"} err="failed to get container status \"312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0\": rpc error: code = NotFound desc = could not find container \"312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0\": container with ID starting with 312dec8e77c7437fc7388a61b0ed4ed7cf2b7e960aea8a7de1edd99a223698d0 not found: ID does not exist" Sep 15 08:00:48.801560 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.801541 2577 scope.go:117] "RemoveContainer" containerID="f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746" Sep 15 08:00:48.801770 ip-10-0-135-38 kubenswrapper[2577]: E0915 08:00:48.801754 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746\": container with ID starting with f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746 not found: ID does not exist" containerID="f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746" Sep 15 08:00:48.801829 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:48.801773 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746"} err="failed to get container status \"f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746\": rpc error: code = NotFound desc = could not find container \"f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746\": container with ID starting with f12d45b19ce7dd83b074c0cd9e46251b0e160dd64eb046593ae861c56ade1746 not found: ID does not exist" Sep 15 08:00:49.949736 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:49.949706 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_847acd2b-7afe-499c-94ae-8a165bfdfa0b/alertmanager/0.log" Sep 15 08:00:49.981430 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:49.981409 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_847acd2b-7afe-499c-94ae-8a165bfdfa0b/config-reloader/0.log" Sep 15 08:00:50.005690 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.005668 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_847acd2b-7afe-499c-94ae-8a165bfdfa0b/kube-rbac-proxy-web/0.log" Sep 15 08:00:50.035195 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.035178 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_847acd2b-7afe-499c-94ae-8a165bfdfa0b/kube-rbac-proxy/0.log" Sep 15 08:00:50.062333 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.062306 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_847acd2b-7afe-499c-94ae-8a165bfdfa0b/kube-rbac-proxy-metric/0.log" Sep 15 08:00:50.089139 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.089120 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_847acd2b-7afe-499c-94ae-8a165bfdfa0b/prom-label-proxy/0.log" Sep 15 08:00:50.115654 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.115637 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_847acd2b-7afe-499c-94ae-8a165bfdfa0b/init-config-reloader/0.log" Sep 15 08:00:50.191823 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.191804 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-f9qt5_6458032e-6e59-405f-908a-a59993c13da8/kube-state-metrics/0.log" Sep 15 08:00:50.217404 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.217383 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-f9qt5_6458032e-6e59-405f-908a-a59993c13da8/kube-rbac-proxy-main/0.log" Sep 15 08:00:50.241065 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.241043 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-f9qt5_6458032e-6e59-405f-908a-a59993c13da8/kube-rbac-proxy-self/0.log" Sep 15 08:00:50.280577 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.280552 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-757d66f764-ms55t_fae5aec8-2fe9-471f-9610-72bbeea8d1bf/metrics-server/0.log" Sep 15 08:00:50.306675 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.306649 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-4v25b_9e5538f3-f1d8-4212-ac70-3f0a85d68f51/monitoring-plugin/0.log" Sep 15 08:00:50.345914 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.345889 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6z2wq_567be93d-745d-4dda-a1e2-44c77d30da63/node-exporter/0.log" Sep 15 08:00:50.369316 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.369300 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6z2wq_567be93d-745d-4dda-a1e2-44c77d30da63/kube-rbac-proxy/0.log" Sep 15 08:00:50.398280 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.398263 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6z2wq_567be93d-745d-4dda-a1e2-44c77d30da63/init-textfile/0.log" Sep 15 08:00:50.597457 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.597376 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-vmnlr_660ea539-a67b-4401-8041-b5cc09913ce0/kube-rbac-proxy-main/0.log" Sep 15 08:00:50.618382 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.618359 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-vmnlr_660ea539-a67b-4401-8041-b5cc09913ce0/kube-rbac-proxy-self/0.log" Sep 15 08:00:50.641961 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.641938 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-vmnlr_660ea539-a67b-4401-8041-b5cc09913ce0/openshift-state-metrics/0.log" Sep 15 08:00:50.679027 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.679005 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa9cb324-dd79-4a3e-8f51-563212ff21d3/prometheus/0.log" Sep 15 08:00:50.706715 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.706691 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa9cb324-dd79-4a3e-8f51-563212ff21d3/config-reloader/0.log" Sep 15 08:00:50.727459 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.727440 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa9cb324-dd79-4a3e-8f51-563212ff21d3/thanos-sidecar/0.log" Sep 15 08:00:50.750899 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.750880 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa9cb324-dd79-4a3e-8f51-563212ff21d3/kube-rbac-proxy-web/0.log" Sep 15 08:00:50.778065 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.778047 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa9cb324-dd79-4a3e-8f51-563212ff21d3/kube-rbac-proxy/0.log" Sep 15 08:00:50.798875 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.798855 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa9cb324-dd79-4a3e-8f51-563212ff21d3/kube-rbac-proxy-thanos/0.log" Sep 15 08:00:50.821632 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.821614 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa9cb324-dd79-4a3e-8f51-563212ff21d3/init-config-reloader/0.log" Sep 15 08:00:50.851155 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.851096 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-qdb5w_875e3815-9b50-4a3b-bb1b-b58ead23ea74/prometheus-operator/0.log" Sep 15 08:00:50.870069 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.870038 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-qdb5w_875e3815-9b50-4a3b-bb1b-b58ead23ea74/kube-rbac-proxy/0.log" Sep 15 08:00:50.893151 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.893129 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-srvxq_30877835-7945-4800-81cb-19ef018d2d72/prometheus-operator-admission-webhook/0.log" Sep 15 08:00:50.930373 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.930352 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74b4cf6f8b-69ktw_efd68fd7-3686-466c-a1b1-4f237e1affc1/telemeter-client/0.log" Sep 15 08:00:50.958150 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.958126 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74b4cf6f8b-69ktw_efd68fd7-3686-466c-a1b1-4f237e1affc1/reload/0.log" Sep 15 08:00:50.995555 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:50.995536 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74b4cf6f8b-69ktw_efd68fd7-3686-466c-a1b1-4f237e1affc1/kube-rbac-proxy/0.log" Sep 15 08:00:51.052802 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:51.052780 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54c44dc67-kpmp5_098e566e-0737-4066-bb61-44a819786dca/thanos-query/0.log" Sep 15 08:00:51.093445 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:51.093424 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54c44dc67-kpmp5_098e566e-0737-4066-bb61-44a819786dca/kube-rbac-proxy-web/0.log" Sep 15 08:00:51.117655 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:51.117591 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54c44dc67-kpmp5_098e566e-0737-4066-bb61-44a819786dca/kube-rbac-proxy/0.log" Sep 15 08:00:51.149154 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:51.149132 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54c44dc67-kpmp5_098e566e-0737-4066-bb61-44a819786dca/prom-label-proxy/0.log" Sep 15 08:00:51.172145 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:51.172124 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54c44dc67-kpmp5_098e566e-0737-4066-bb61-44a819786dca/kube-rbac-proxy-rules/0.log" Sep 15 08:00:51.196552 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:51.196537 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54c44dc67-kpmp5_098e566e-0737-4066-bb61-44a819786dca/kube-rbac-proxy-metrics/0.log" Sep 15 08:00:53.034403 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.034356 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8"] Sep 15 08:00:53.034843 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.034710 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" containerName="copy" Sep 15 08:00:53.034843 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.034721 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" containerName="copy" Sep 15 08:00:53.034843 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.034732 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" containerName="gather" Sep 15 08:00:53.034843 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.034737 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" containerName="gather" Sep 15 08:00:53.034843 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.034810 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" containerName="copy" Sep 15 08:00:53.034843 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.034820 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="496a83f0-ee0a-4217-9f01-44cc90ff2f6c" containerName="gather" Sep 15 08:00:53.038095 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.038077 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.042330 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.042306 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p2wd7\"/\"default-dockercfg-p2kfg\"" Sep 15 08:00:53.042460 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.042310 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p2wd7\"/\"kube-root-ca.crt\"" Sep 15 08:00:53.042460 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.042377 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p2wd7\"/\"openshift-service-ca.crt\"" Sep 15 08:00:53.046566 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.046542 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8"] Sep 15 08:00:53.146157 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.146120 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-sys\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.146368 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.146201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-lib-modules\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.146368 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.146230 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-proc\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.146368 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.146255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-podres\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.146368 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.146283 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f8wm\" (UniqueName: \"kubernetes.io/projected/be447262-a397-4691-aae8-875d09df1469-kube-api-access-5f8wm\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.153434 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.153413 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-vzq75_9bf86a8e-64a0-4edf-be9b-1543f8ab172a/download-server/0.log" Sep 15 08:00:53.247112 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.247072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-sys\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.247276 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.247134 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-lib-modules\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.247276 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.247153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-proc\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.247276 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.247174 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-podres\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.247276 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.247198 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f8wm\" (UniqueName: \"kubernetes.io/projected/be447262-a397-4691-aae8-875d09df1469-kube-api-access-5f8wm\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.247444 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.247281 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-proc\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.247444 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.247301 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-lib-modules\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.247444 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.247319 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-podres\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.247444 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.247301 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be447262-a397-4691-aae8-875d09df1469-sys\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.255849 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.255828 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f8wm\" (UniqueName: \"kubernetes.io/projected/be447262-a397-4691-aae8-875d09df1469-kube-api-access-5f8wm\") pod \"perf-node-gather-daemonset-45xc8\" (UID: \"be447262-a397-4691-aae8-875d09df1469\") " pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.347951 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.347882 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.614696 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.614669 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8"] Sep 15 08:00:53.617174 ip-10-0-135-38 kubenswrapper[2577]: W0915 08:00:53.617141 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbe447262_a397_4691_aae8_875d09df1469.slice/crio-372da2cb9fed300e962c549c91f39d7a060701de939bb40e26d9b831a225fec4 WatchSource:0}: Error finding container 372da2cb9fed300e962c549c91f39d7a060701de939bb40e26d9b831a225fec4: Status 404 returned error can't find the container with id 372da2cb9fed300e962c549c91f39d7a060701de939bb40e26d9b831a225fec4 Sep 15 08:00:53.804896 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.804864 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" event={"ID":"be447262-a397-4691-aae8-875d09df1469","Type":"ContainerStarted","Data":"e6bf13bfe354454fc38cf8081d1487b9021c2ae82f239845f00f4481432d01cd"} Sep 15 08:00:53.805012 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.804922 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" event={"ID":"be447262-a397-4691-aae8-875d09df1469","Type":"ContainerStarted","Data":"372da2cb9fed300e962c549c91f39d7a060701de939bb40e26d9b831a225fec4"} Sep 15 08:00:53.805012 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.804973 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:00:53.822992 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:53.822953 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" podStartSLOduration=0.822938109 podStartE2EDuration="822.938109ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 08:00:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 08:00:53.821594404 +0000 UTC m=+2145.644065810" watchObservedRunningTime="2026-09-15 08:00:53.822938109 +0000 UTC m=+2145.645409514" Sep 15 08:00:54.405610 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:54.405579 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fhqmn_0916e3c6-af2a-4b29-8f5f-b4eca5a04b77/dns/0.log" Sep 15 08:00:54.422248 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:54.422226 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fhqmn_0916e3c6-af2a-4b29-8f5f-b4eca5a04b77/kube-rbac-proxy/0.log" Sep 15 08:00:54.483265 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:54.483232 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-96dhs_5b7c5265-0d16-4645-8524-5c3ba1839641/dns-node-resolver/0.log" Sep 15 08:00:54.953728 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:54.953700 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k8pc6_3e78afdb-75af-4944-a28b-7850f145d165/node-ca/0.log" Sep 15 08:00:55.891455 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:55.891422 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6msdh_f8933686-b777-4397-bcaa-b0ce3f7f4a9e/serve-healthcheck-canary/0.log" Sep 15 08:00:56.262647 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:56.262618 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6xrpp_5b36d217-aaff-4fa0-bfa6-11a85b51b1bc/kube-rbac-proxy/0.log" Sep 15 08:00:56.282199 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:56.282178 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6xrpp_5b36d217-aaff-4fa0-bfa6-11a85b51b1bc/exporter/0.log" Sep 15 08:00:56.301523 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:56.301506 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6xrpp_5b36d217-aaff-4fa0-bfa6-11a85b51b1bc/extractor/0.log" Sep 15 08:00:59.818216 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:00:59.818185 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p2wd7/perf-node-gather-daemonset-45xc8" Sep 15 08:01:01.393369 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:01.393338 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2vds4_c14334f4-fee6-411c-a8a4-228a183a7da2/kube-multus/0.log" Sep 15 08:01:01.443145 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:01.443116 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22spd_f4c07647-b791-48b8-bd5e-01de110142d2/kube-multus-additional-cni-plugins/0.log" Sep 15 08:01:01.464840 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:01.464813 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22spd_f4c07647-b791-48b8-bd5e-01de110142d2/egress-router-binary-copy/0.log" Sep 15 08:01:01.488490 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:01.488455 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22spd_f4c07647-b791-48b8-bd5e-01de110142d2/cni-plugins/0.log" Sep 15 08:01:01.512859 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:01.512834 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22spd_f4c07647-b791-48b8-bd5e-01de110142d2/bond-cni-plugin/0.log" Sep 15 08:01:01.535833 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:01.535810 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22spd_f4c07647-b791-48b8-bd5e-01de110142d2/routeoverride-cni/0.log" Sep 15 08:01:01.556547 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:01.556530 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22spd_f4c07647-b791-48b8-bd5e-01de110142d2/whereabouts-cni-bincopy/0.log" Sep 15 08:01:01.577279 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:01.577258 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22spd_f4c07647-b791-48b8-bd5e-01de110142d2/whereabouts-cni/0.log" Sep 15 08:01:02.037633 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:02.037552 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f9mmh_7ef119f1-8070-4d2a-b511-1fd19295935b/network-metrics-daemon/0.log" Sep 15 08:01:02.059937 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:02.059913 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f9mmh_7ef119f1-8070-4d2a-b511-1fd19295935b/kube-rbac-proxy/0.log" Sep 15 08:01:02.879604 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:02.879577 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4k5xc_aaef9236-1230-4c5a-9f06-08458e473e03/ovn-controller/0.log" Sep 15 08:01:02.897709 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:02.897686 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4k5xc_aaef9236-1230-4c5a-9f06-08458e473e03/ovn-acl-logging/0.log" Sep 15 08:01:02.919859 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:02.919835 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4k5xc_aaef9236-1230-4c5a-9f06-08458e473e03/kube-rbac-proxy-node/0.log" Sep 15 08:01:02.942066 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:02.942044 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4k5xc_aaef9236-1230-4c5a-9f06-08458e473e03/kube-rbac-proxy-ovn-metrics/0.log" Sep 15 08:01:02.964397 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:02.964374 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4k5xc_aaef9236-1230-4c5a-9f06-08458e473e03/northd/0.log" Sep 15 08:01:02.988712 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:02.988692 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4k5xc_aaef9236-1230-4c5a-9f06-08458e473e03/nbdb/0.log" Sep 15 08:01:03.010430 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:03.010409 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4k5xc_aaef9236-1230-4c5a-9f06-08458e473e03/sbdb/0.log" Sep 15 08:01:03.049140 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:03.049115 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4k5xc_aaef9236-1230-4c5a-9f06-08458e473e03/ovnkube-controller/0.log" Sep 15 08:01:04.410874 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:04.410845 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-m5skf_735140f1-e909-449e-8129-be3752b71965/network-check-target-container/0.log" Sep 15 08:01:05.279082 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:05.279055 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qmdkl_20607e49-0155-468e-b8d4-0fda61b3be41/iptables-alerter/0.log" Sep 15 08:01:05.830035 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:05.830006 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-8zr7v_61db389e-742a-4164-9417-d95ca886cc78/tuned/0.log" Sep 15 08:01:09.030585 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:09.030512 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-kkrst_98080ddc-50f4-48a0-a0ef-59f24adf7ea1/csi-driver/0.log" Sep 15 08:01:09.050863 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:09.050841 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-kkrst_98080ddc-50f4-48a0-a0ef-59f24adf7ea1/csi-node-driver-registrar/0.log" Sep 15 08:01:09.078496 ip-10-0-135-38 kubenswrapper[2577]: I0915 08:01:09.078465 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-kkrst_98080ddc-50f4-48a0-a0ef-59f24adf7ea1/csi-liveness-probe/0.log"