Sep 10 15:55:57.119966 ip-10-0-129-74 systemd[1]: Starting Kubernetes Kubelet... Sep 10 15:55:57.475671 ip-10-0-129-74 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 15:55:57.475671 ip-10-0-129-74 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 15:55:57.475671 ip-10-0-129-74 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 15:55:57.475671 ip-10-0-129-74 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 15:55:57.475671 ip-10-0-129-74 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 15:55:57.476547 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.476445 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 15:55:57.479502 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479487 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 15:55:57.479502 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479502 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479506 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479510 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479513 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479516 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479518 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479521 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479524 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479526 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479529 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479533 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479545 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479548 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479551 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479554 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479557 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479560 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479563 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479565 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 15:55:57.479564 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479568 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479571 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479574 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479577 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479581 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479584 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479587 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479590 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479593 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479595 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479598 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479601 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479603 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479606 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479609 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479611 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479614 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479616 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479619 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479621 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 15:55:57.480085 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479624 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479626 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479630 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479633 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479635 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479638 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479640 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479643 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479646 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479648 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479650 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479653 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479655 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479657 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479661 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479664 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479667 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479669 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479672 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479675 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 15:55:57.480571 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479677 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479680 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479682 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479685 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479687 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479690 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479692 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479695 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479697 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479700 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479702 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479717 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479721 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479724 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479726 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479729 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479732 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479734 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479737 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479739 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 15:55:57.481078 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479742 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479745 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479749 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479753 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479756 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479759 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479762 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479765 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479768 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479772 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479775 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479777 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479780 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479783 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479786 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479788 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479791 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479794 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 15:55:57.481576 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.479796 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.481998 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482006 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482009 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482013 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482016 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482020 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482023 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482026 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482029 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482032 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482034 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482037 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482040 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482043 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 15:55:57.482039 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482046 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482049 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482052 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482055 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482057 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482060 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482062 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482065 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482067 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482071 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482075 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482078 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482080 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482083 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482085 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482088 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482090 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482093 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482095 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 15:55:57.482413 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482098 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482100 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482102 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482105 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482107 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482110 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482113 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482115 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482118 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482120 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482123 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482125 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482128 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482130 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482133 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482137 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482139 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482142 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482145 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482147 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 15:55:57.482897 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482150 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482153 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482155 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482158 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482161 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482163 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482165 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482168 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482170 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482173 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482175 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482179 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482182 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482185 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482189 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482192 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482195 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482198 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482201 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482204 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 15:55:57.483397 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482207 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482210 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482212 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482215 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482218 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482221 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482224 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482227 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482230 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482232 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482235 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482237 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482240 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482242 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482245 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482247 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482250 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482254 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482256 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482259 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 15:55:57.483896 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482261 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482264 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482266 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482269 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482273 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482276 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482664 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482669 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482672 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482675 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482679 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482682 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482685 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482688 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482690 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482693 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482696 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482698 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482702 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 15:55:57.484388 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482718 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482721 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482725 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482728 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482731 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482734 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482737 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482740 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482743 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482745 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482749 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482752 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482754 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482757 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482759 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482762 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482766 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482769 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482772 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 15:55:57.484862 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482775 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482778 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482780 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482783 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482785 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482789 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482791 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482794 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482797 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482799 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482802 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482805 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482807 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482810 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482812 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482815 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482818 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482820 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482823 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482825 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 15:55:57.485341 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482828 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482830 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482833 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482835 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482837 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482840 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482842 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482845 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482847 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482852 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482854 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482857 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482860 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482862 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482865 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482867 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482870 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482872 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482875 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482877 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 15:55:57.485946 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482881 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482883 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482886 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482888 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482891 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482894 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482897 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482899 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482902 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482905 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482907 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482910 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482913 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482916 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482918 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482921 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482923 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482926 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482928 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482931 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 15:55:57.486427 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482933 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482936 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482939 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482942 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482944 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482947 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.482950 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483015 2576 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483057 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483066 2576 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483072 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483077 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483081 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483085 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483089 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483092 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483095 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483098 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483102 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483105 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483109 2576 flags.go:64] FLAG: --cgroup-root="" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483111 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 15:55:57.486938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483114 2576 flags.go:64] FLAG: --client-ca-file="" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483117 2576 flags.go:64] FLAG: --cloud-provider="external" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483120 2576 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483124 2576 flags.go:64] FLAG: --cluster-domain="" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483127 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483129 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483132 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483135 2576 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483139 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483142 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483145 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483148 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483151 2576 flags.go:64] FLAG: --contention-profiling="false" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483154 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483157 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483160 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483162 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483166 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483169 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483172 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483175 2576 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483177 2576 flags.go:64] FLAG: --enable-server="true" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483181 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483184 2576 flags.go:64] FLAG: --event-burst="100" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483187 2576 flags.go:64] FLAG: --event-qps="50" Sep 10 15:55:57.487492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483190 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483196 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483199 2576 flags.go:64] FLAG: --eviction-hard="" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483202 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483205 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483208 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483211 2576 flags.go:64] FLAG: --eviction-soft="" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483214 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483217 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483219 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483222 2576 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483225 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483228 2576 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483231 2576 flags.go:64] FLAG: --feature-gates="" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483234 2576 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483237 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483240 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483243 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483246 2576 flags.go:64] FLAG: --healthz-port="10248" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483249 2576 flags.go:64] FLAG: --help="false" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483252 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483255 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483258 2576 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 15:55:57.488105 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483261 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483264 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483267 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483270 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483272 2576 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483275 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483278 2576 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483281 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483285 2576 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483288 2576 flags.go:64] FLAG: --kube-reserved="" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483292 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483294 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483297 2576 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483300 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483303 2576 flags.go:64] FLAG: --lock-file="" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483306 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483309 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483311 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483319 2576 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483322 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483325 2576 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483328 2576 flags.go:64] FLAG: --logging-format="text" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483331 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483334 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 15:55:57.488684 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483337 2576 flags.go:64] FLAG: --manifest-url="" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483340 2576 flags.go:64] FLAG: --manifest-url-header="" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483344 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483347 2576 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483351 2576 flags.go:64] FLAG: --max-pods="110" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483354 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483357 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483360 2576 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483363 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483366 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483369 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483371 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483378 2576 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483381 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483384 2576 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483387 2576 flags.go:64] FLAG: --pod-cidr="" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483390 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483395 2576 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483398 2576 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483402 2576 flags.go:64] FLAG: --pods-per-core="0" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483405 2576 flags.go:64] FLAG: --port="10250" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483409 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483412 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ee9f01ba22cb61f1" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483415 2576 flags.go:64] FLAG: --qos-reserved="" Sep 10 15:55:57.489305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483417 2576 flags.go:64] FLAG: --read-only-port="10255" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483420 2576 flags.go:64] FLAG: --register-node="true" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483423 2576 flags.go:64] FLAG: --register-with-taints="" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483427 2576 flags.go:64] FLAG: --registry-burst="10" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483430 2576 flags.go:64] FLAG: --registry-qps="5" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483432 2576 flags.go:64] FLAG: --reserved-cpus="" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483435 2576 flags.go:64] FLAG: --reserved-memory="" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483438 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483441 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483444 2576 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483447 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483452 2576 flags.go:64] FLAG: --runonce="false" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483455 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483458 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483460 2576 flags.go:64] FLAG: --seccomp-default="false" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483463 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483466 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483469 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483472 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483475 2576 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483477 2576 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483480 2576 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483483 2576 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483486 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483489 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 15:55:57.489912 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483492 2576 flags.go:64] FLAG: --system-cgroups="" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483495 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483501 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483504 2576 flags.go:64] FLAG: --tls-cert-file="" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483506 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483511 2576 flags.go:64] FLAG: --tls-min-version="" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483513 2576 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483516 2576 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483519 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483521 2576 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483524 2576 flags.go:64] FLAG: --v="2" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483528 2576 flags.go:64] FLAG: --version="false" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483532 2576 flags.go:64] FLAG: --vmodule="" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483535 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.483538 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483650 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483653 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483657 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483663 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483666 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483669 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483673 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 15:55:57.490568 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483675 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483678 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483680 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483683 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483685 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483688 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483690 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483693 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483696 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483698 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483702 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483723 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483728 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483731 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483733 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483736 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483739 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483741 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483744 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 15:55:57.491127 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483746 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483749 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483752 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483755 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483757 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483760 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483762 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483765 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483767 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483771 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483774 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483777 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483779 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483782 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483785 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483788 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483790 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483793 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483795 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483798 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 15:55:57.491590 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483800 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483803 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483805 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483808 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483811 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483820 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483823 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483826 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483828 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483831 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483833 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483837 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483841 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483844 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483848 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483850 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483853 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483856 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483858 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483861 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 15:55:57.492089 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483864 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483868 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483871 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483873 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483875 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483878 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483881 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483883 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483886 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483888 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483891 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483893 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483896 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483899 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483901 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483904 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483906 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483910 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483914 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483917 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 15:55:57.492614 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483919 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483922 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483924 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483927 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483929 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483932 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483934 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483937 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483939 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483942 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483944 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483947 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.483950 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 15:55:57.493124 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.484508 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.493274 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.493286 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.493298 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.493303 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493361 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493366 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493369 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493373 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493375 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493378 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493381 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493383 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493386 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493388 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493391 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493394 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493397 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493400 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 15:55:57.493457 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493403 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493405 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493408 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493411 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493414 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493417 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493419 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493422 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493424 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493427 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493430 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493432 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493435 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493438 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493441 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493444 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493447 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493450 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493452 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493455 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493458 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 15:55:57.493941 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493461 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493463 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493466 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493469 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493471 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493474 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493476 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493480 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493484 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493487 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493491 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493493 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493496 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493498 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493501 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493503 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493506 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493509 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493512 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493514 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 15:55:57.494450 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493517 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493519 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493522 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493524 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493527 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493529 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493532 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493536 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493539 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493543 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493546 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493549 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493552 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493555 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493557 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493559 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493562 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493565 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493567 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 15:55:57.494959 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493570 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493572 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493576 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493579 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493582 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493585 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493588 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493591 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493593 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493596 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493599 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493601 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493603 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493606 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493609 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493611 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493614 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493616 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493619 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 15:55:57.495420 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493622 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493624 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493627 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493630 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493632 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493635 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.493639 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493773 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493779 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493782 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493785 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493788 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493791 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493794 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493796 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 15:55:57.495932 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493799 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493802 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493805 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493808 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493810 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493813 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493816 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493818 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493821 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493823 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493826 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493828 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493831 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493833 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493836 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493838 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493841 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493844 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493846 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 15:55:57.496307 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493849 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493852 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493856 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493859 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493861 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493864 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493867 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493869 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493871 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493874 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493876 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493879 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493881 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493884 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493886 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493890 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493893 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493897 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493899 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493902 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 15:55:57.496781 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493905 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493908 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493911 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493913 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493916 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493919 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493921 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493923 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493926 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493928 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493931 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493934 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493936 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493938 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493941 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493943 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493946 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493949 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493951 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493954 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 15:55:57.497281 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493956 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493959 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493962 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493964 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493966 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493969 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493973 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493976 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493978 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493980 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493983 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493986 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493988 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493991 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493993 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493996 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.493998 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494001 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494003 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 15:55:57.497771 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494006 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494008 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494011 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494015 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494018 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494020 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494023 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494025 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494028 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494030 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494033 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494036 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:57.494040 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.494044 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 15:55:57.498230 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.494133 2576 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 15:55:57.498566 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.497606 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 15:55:57.498566 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.498551 2576 server.go:1019] "Starting client certificate rotation" Sep 10 15:55:57.498668 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.498648 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 15:55:57.498697 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.498689 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 15:55:57.501550 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.501537 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 15:55:57.507299 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.507280 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 15:55:57.516802 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.516781 2576 log.go:25] "Validated CRI v1 runtime API" Sep 10 15:55:57.519899 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.519886 2576 log.go:25] "Validated CRI v1 image API" Sep 10 15:55:57.520785 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.520767 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 15:55:57.525931 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.525913 2576 fs.go:135] Filesystem UUIDs: map[3e2b0209-1210-44fa-9203-b4c2bf01afb7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 83e8f1fd-9092-48b9-bbee-b836a741c578:/dev/nvme0n1p3] Sep 10 15:55:57.526006 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.525929 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 15:55:57.529328 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.529232 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 15:55:57.530653 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.530553 2576 manager.go:216] Machine: {Timestamp:2026-09-10 15:55:57.529324985 +0000 UTC m=+0.305383408 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3116763 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ada5cd5f31ac1705efe96c8d46194 SystemUUID:ec2ada5c-d5f3-1ac1-705e-fe96c8d46194 BootID:c1c9189a-4c22-474b-bf32-95b1939d257f Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e6:30:b3:a9:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e6:30:b3:a9:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:a9:7f:4b:d7:9e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 15:55:57.530653 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.530652 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 15:55:57.530774 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.530733 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 15:55:57.533717 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.533686 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 15:55:57.533867 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.533718 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-74.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 15:55:57.534736 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.534726 2576 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 15:55:57.534769 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.534739 2576 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 15:55:57.534769 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.534746 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 15:55:57.535399 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.535389 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 15:55:57.537001 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.536991 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 15:55:57.541880 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.541868 2576 state_mem.go:36] "Initialized new in-memory state store" Sep 10 15:55:57.541998 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.541989 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 15:55:57.542038 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.542016 2576 kubelet.go:486] "Attempting to sync node with API server" Sep 10 15:55:57.542038 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.542035 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 15:55:57.542109 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.542048 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 15:55:57.542109 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.542057 2576 kubelet.go:398] "Adding apiserver pod source" Sep 10 15:55:57.542109 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.542065 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 15:55:57.542381 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.542366 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 15:55:57.542435 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.542392 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 15:55:57.545250 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.545233 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 15:55:57.547954 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.547941 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 15:55:57.548003 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.547970 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 15:55:57.548780 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548769 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 15:55:57.548814 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548785 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 15:55:57.548814 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548790 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 15:55:57.548814 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548795 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 15:55:57.548814 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548805 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 15:55:57.548814 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548811 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 15:55:57.548946 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548825 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 15:55:57.548946 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548831 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 15:55:57.548946 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548837 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 15:55:57.548946 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548843 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 15:55:57.548946 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548856 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 15:55:57.548946 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.548862 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 15:55:57.549615 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.549604 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 15:55:57.549648 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.549618 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 15:55:57.552058 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.552040 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-74.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 15:55:57.552572 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.552543 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 15:55:57.552688 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.552660 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 15:55:57.553811 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.553796 2576 server.go:1268] "Started kubelet" Sep 10 15:55:57.553914 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.553889 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 15:55:57.553997 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.553950 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 15:55:57.554038 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.554018 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 15:55:57.554620 ip-10-0-129-74 systemd[1]: Started Kubernetes Kubelet. Sep 10 15:55:57.555490 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.555472 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 15:55:57.555788 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.555775 2576 server.go:310] "Adding debug handlers to kubelet server" Sep 10 15:55:57.556388 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.556367 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bdmxt" Sep 10 15:55:57.557674 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.557653 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 15:55:57.560180 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.560162 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 15:55:57.560180 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.560167 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 15:55:57.560452 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.560332 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:57.560822 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.560802 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 15:55:57.560822 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.560819 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 15:55:57.560945 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.560801 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 15:55:57.560945 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.560900 2576 reconstruct.go:97] "Volume reconstruction finished" Sep 10 15:55:57.560945 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.560908 2576 reconciler.go:29] "Reconciler: start to sync state" Sep 10 15:55:57.565599 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.565583 2576 factory.go:55] Registering systemd factory Sep 10 15:55:57.565599 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.565601 2576 factory.go:223] Registration of the systemd container factory successfully Sep 10 15:55:57.566088 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.566073 2576 factory.go:153] Registering CRI-O factory Sep 10 15:55:57.566088 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.566089 2576 factory.go:223] Registration of the crio container factory successfully Sep 10 15:55:57.566227 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.566137 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 15:55:57.566227 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.566152 2576 factory.go:103] Registering Raw factory Sep 10 15:55:57.566227 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.566167 2576 manager.go:1196] Started watching for new ooms in manager Sep 10 15:55:57.566538 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.566511 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bdmxt" Sep 10 15:55:57.566932 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.566908 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 15:55:57.566932 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.566920 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 15:55:57.567062 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.567054 2576 manager.go:349] Starting recovery of all containers Sep 10 15:55:57.567947 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.567104 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-74.ec2.internal.18d400bb905c1f8e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-74.ec2.internal,UID:ip-10-0-129-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-74.ec2.internal,},FirstTimestamp:2026-09-10 15:55:57.553770382 +0000 UTC m=+0.329828804,LastTimestamp:2026-09-10 15:55:57.553770382 +0000 UTC m=+0.329828804,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-74.ec2.internal,}" Sep 10 15:55:57.568225 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.568207 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 15:55:57.577920 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.577899 2576 manager.go:354] Recovery completed Sep 10 15:55:57.582811 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.582798 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 15:55:57.584554 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.584535 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 15:55:57.584631 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.584562 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 15:55:57.584631 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.584572 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 15:55:57.585075 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.585062 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 15:55:57.585075 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.585073 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 15:55:57.585184 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.585087 2576 state_mem.go:36] "Initialized new in-memory state store" Sep 10 15:55:57.585184 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.585073 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 15:55:57.585920 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.585898 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 15:55:57.586008 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.585930 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 15:55:57.586008 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.585946 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 15:55:57.586573 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.586507 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-74.ec2.internal.18d400bb9231ca38 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-74.ec2.internal,UID:ip-10-0-129-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-74.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-74.ec2.internal,},FirstTimestamp:2026-09-10 15:55:57.584550456 +0000 UTC m=+0.360608879,LastTimestamp:2026-09-10 15:55:57.584550456 +0000 UTC m=+0.360608879,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-74.ec2.internal,}" Sep 10 15:55:57.587077 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.587063 2576 policy_none.go:49] "None policy: Start" Sep 10 15:55:57.587077 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.587079 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 15:55:57.587184 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.587089 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 15:55:57.589075 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.589061 2576 policy_none.go:47] "Start" Sep 10 15:55:57.615491 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.615477 2576 manager.go:337] "Starting Device Plugin manager" Sep 10 15:55:57.641629 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.615515 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 15:55:57.641629 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.615526 2576 server.go:87] "Starting device plugin registration server" Sep 10 15:55:57.641629 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.615853 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 15:55:57.641629 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.615865 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 15:55:57.641629 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.615990 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 15:55:57.641629 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.616060 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 15:55:57.641629 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.616066 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 15:55:57.641629 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.617201 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 15:55:57.641629 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.617254 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:57.693145 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.693129 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 15:55:57.693246 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.693181 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 15:55:57.693246 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.693209 2576 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 15:55:57.693349 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.693258 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 15:55:57.695860 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.695842 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 15:55:57.715968 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.715944 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 15:55:57.716738 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.716722 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 15:55:57.716817 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.716749 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 15:55:57.716817 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.716758 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 15:55:57.716817 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.716780 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.725461 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.725445 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.725542 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.725469 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-74.ec2.internal\": node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:57.742861 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.742795 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:57.793787 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.793765 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal"] Sep 10 15:55:57.793865 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.793836 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 15:55:57.794638 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.794624 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 15:55:57.794699 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.794649 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 15:55:57.794699 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.794659 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 15:55:57.796815 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.796802 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 15:55:57.796928 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.796914 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.796983 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.796946 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 15:55:57.797583 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.797563 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 15:55:57.797658 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.797563 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 15:55:57.797658 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.797618 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 15:55:57.797658 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.797630 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 15:55:57.797658 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.797593 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 15:55:57.797788 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.797661 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 15:55:57.799768 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.799751 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.799825 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.799783 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 15:55:57.800443 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.800426 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 15:55:57.800523 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.800454 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 15:55:57.800523 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.800469 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 15:55:57.823292 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.823269 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-74.ec2.internal\" not found" node="ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.827601 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.827586 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-74.ec2.internal\" not found" node="ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.843505 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.843492 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:57.862944 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.862927 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae9bd1aaa63cabc2deea36246571659b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal\" (UID: \"ae9bd1aaa63cabc2deea36246571659b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.862989 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.862950 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e59b8d96466349d07517a47588dc5f77-config\") pod \"kube-apiserver-proxy-ip-10-0-129-74.ec2.internal\" (UID: \"e59b8d96466349d07517a47588dc5f77\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.862989 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.862966 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae9bd1aaa63cabc2deea36246571659b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal\" (UID: \"ae9bd1aaa63cabc2deea36246571659b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.943808 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:57.943792 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:57.963980 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.963962 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e59b8d96466349d07517a47588dc5f77-config\") pod \"kube-apiserver-proxy-ip-10-0-129-74.ec2.internal\" (UID: \"e59b8d96466349d07517a47588dc5f77\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.964026 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.963985 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae9bd1aaa63cabc2deea36246571659b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal\" (UID: \"ae9bd1aaa63cabc2deea36246571659b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.964026 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.964001 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae9bd1aaa63cabc2deea36246571659b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal\" (UID: \"ae9bd1aaa63cabc2deea36246571659b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.964087 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.964025 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae9bd1aaa63cabc2deea36246571659b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal\" (UID: \"ae9bd1aaa63cabc2deea36246571659b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.964087 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.964054 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e59b8d96466349d07517a47588dc5f77-config\") pod \"kube-apiserver-proxy-ip-10-0-129-74.ec2.internal\" (UID: \"e59b8d96466349d07517a47588dc5f77\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal" Sep 10 15:55:57.964087 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:57.964057 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae9bd1aaa63cabc2deea36246571659b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal\" (UID: \"ae9bd1aaa63cabc2deea36246571659b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" Sep 10 15:55:58.044329 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.044289 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:58.126857 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.126830 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal" Sep 10 15:55:58.131852 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.131733 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" Sep 10 15:55:58.144759 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.144721 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:58.245265 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.245229 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:58.345782 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.345717 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:58.446731 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.446699 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:58.453930 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.453908 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 15:55:58.498320 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.498299 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 15:55:58.498716 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.498408 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 15:55:58.498716 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.498461 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 15:55:58.547840 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.547797 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:58.570310 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.570278 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 15:50:58 +0000 UTC" deadline="2028-06-27 20:59:44.70029452 +0000 UTC" Sep 10 15:55:58.570310 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.570308 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15749h3m46.129989208s" Sep 10 15:55:58.648767 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.648741 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:58.749378 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.749347 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:58.771251 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:58.771095 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59b8d96466349d07517a47588dc5f77.slice/crio-15672d867c627414c17ce5eb8aa77f07a0946bf6c6b1cca64b76c7e72ff7fd9c WatchSource:0}: Error finding container 15672d867c627414c17ce5eb8aa77f07a0946bf6c6b1cca64b76c7e72ff7fd9c: Status 404 returned error can't find the container with id 15672d867c627414c17ce5eb8aa77f07a0946bf6c6b1cca64b76c7e72ff7fd9c Sep 10 15:55:58.775518 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.775503 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 15:55:58.798460 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:55:58.798437 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae9bd1aaa63cabc2deea36246571659b.slice/crio-80ad983c167c1796dfce76584c22ae28e189b573e27dad0af2289c2c5b15a311 WatchSource:0}: Error finding container 80ad983c167c1796dfce76584c22ae28e189b573e27dad0af2289c2c5b15a311: Status 404 returned error can't find the container with id 80ad983c167c1796dfce76584c22ae28e189b573e27dad0af2289c2c5b15a311 Sep 10 15:55:58.850310 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.850281 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:58.917027 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:58.916964 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 15:55:58.952767 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:58.952744 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-74.ec2.internal\" not found" Sep 10 15:55:59.036685 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.036665 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 15:55:59.061196 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.061177 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal" Sep 10 15:55:59.075100 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.075084 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 15:55:59.076038 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.076019 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" Sep 10 15:55:59.086542 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.086524 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 15:55:59.543267 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.543232 2576 apiserver.go:52] "Watching apiserver" Sep 10 15:55:59.552165 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.552139 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 15:55:59.552757 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.552692 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal","openshift-multus/multus-additional-cni-plugins-nq9fq","openshift-multus/network-metrics-daemon-jclb4","openshift-ovn-kubernetes/ovnkube-node-7zwpx","kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn","openshift-dns/node-resolver-86g67","openshift-image-registry/node-ca-sms7n","openshift-multus/multus-4kfq7","openshift-network-diagnostics/network-check-target-mwvrc","openshift-network-operator/iptables-alerter-6gf2s","kube-system/global-pull-secret-syncer-q49dm","kube-system/konnectivity-agent-w9mp9","openshift-cluster-node-tuning-operator/tuned-tdhmk"] Sep 10 15:55:59.555639 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.555615 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.557914 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.557887 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.559041 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.559017 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 15:55:59.559242 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.559224 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 15:55:59.559510 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.559492 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pt7ns\"" Sep 10 15:55:59.560303 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.560281 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:55:59.560476 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.560441 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:55:59.562144 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.562123 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 15:55:59.562571 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.562547 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 15:55:59.562717 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.562699 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 15:55:59.562923 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.562904 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 15:55:59.563004 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.562914 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gmdxz\"" Sep 10 15:55:59.563234 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.563216 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.565564 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.565542 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.565929 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.565899 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 15:55:59.566128 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.566106 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l9fl6\"" Sep 10 15:55:59.567577 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.567553 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 15:55:59.567674 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.567619 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 15:55:59.567839 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.567804 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 15:55:59.568686 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.567957 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 15:55:59.568686 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.568037 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 15:55:59.568686 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.568328 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 15:55:59.568686 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.568542 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xpddk\"" Sep 10 15:55:59.568686 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.568556 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 15:55:59.568959 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.568726 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.571302 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.571281 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.572466 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.572445 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 15:55:59.572626 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.572608 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hftbp\"" Sep 10 15:55:59.572739 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.572691 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 15:55:59.572830 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.572697 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 15:55:59.574326 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.574307 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nnjv8\"" Sep 10 15:55:59.574429 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.574329 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:55:59.574572 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.574554 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 15:55:59.574730 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.574712 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 15:55:59.574970 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.574950 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 15:55:59.575044 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.574960 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 15:55:59.575344 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575163 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-ovnkube-config\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.575344 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-sysconfig\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.575344 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575223 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a39261f-d1f2-438d-a6ef-b83066bbead7-tmp-dir\") pod \"node-resolver-86g67\" (UID: \"0a39261f-d1f2-438d-a6ef-b83066bbead7\") " pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.575344 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575250 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqw4j\" (UniqueName: \"kubernetes.io/projected/5c40402d-c9b9-4d90-9d8f-1af6003a16e5-kube-api-access-fqw4j\") pod \"iptables-alerter-6gf2s\" (UID: \"5c40402d-c9b9-4d90-9d8f-1af6003a16e5\") " pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.575344 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-sysctl-conf\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.575344 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575316 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-systemd\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575348 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-tuned\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575420 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c29a7638-5037-4b0e-91f8-074b80b0bb61-tmp\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575450 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwds2\" (UniqueName: \"kubernetes.io/projected/c29a7638-5037-4b0e-91f8-074b80b0bb61-kube-api-access-hwds2\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575477 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-log-socket\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575521 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a39261f-d1f2-438d-a6ef-b83066bbead7-hosts-file\") pod \"node-resolver-86g67\" (UID: \"0a39261f-d1f2-438d-a6ef-b83066bbead7\") " pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575562 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2lnd\" (UniqueName: \"kubernetes.io/projected/0a39261f-d1f2-438d-a6ef-b83066bbead7-kube-api-access-m2lnd\") pod \"node-resolver-86g67\" (UID: \"0a39261f-d1f2-438d-a6ef-b83066bbead7\") " pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575603 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575633 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-run\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575676 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5c40402d-c9b9-4d90-9d8f-1af6003a16e5-iptables-alerter-script\") pod \"iptables-alerter-6gf2s\" (UID: \"5c40402d-c9b9-4d90-9d8f-1af6003a16e5\") " pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575702 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-kubelet\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.575864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575858 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-slash\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575885 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-etc-openvswitch\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575910 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-ovn-node-metrics-cert\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575933 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t2zn\" (UniqueName: \"kubernetes.io/projected/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-kube-api-access-5t2zn\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575957 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-run-systemd\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.575980 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-run-ovn\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576005 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-node-log\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576028 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-sysctl-d\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576056 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-sys\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576079 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-run-netns\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576103 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-var-lib-openvswitch\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576168 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-systemd-units\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576193 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-cni-netd\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576217 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-kubernetes\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576241 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-lib-modules\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.576446 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576286 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-host\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576311 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgnm9\" (UniqueName: \"kubernetes.io/projected/43c09421-c94c-4ec6-abe4-a30be09975c6-kube-api-access-fgnm9\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576336 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576361 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-cni-bin\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576386 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-env-overrides\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576420 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-ovnkube-script-lib\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576446 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-modprobe-d\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576471 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-var-lib-kubelet\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576495 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c40402d-c9b9-4d90-9d8f-1af6003a16e5-host-slash\") pod \"iptables-alerter-6gf2s\" (UID: \"5c40402d-c9b9-4d90-9d8f-1af6003a16e5\") " pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.576521 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-run-openvswitch\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.577502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.577245 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.578380 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.578184 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4zddg\"" Sep 10 15:55:59.578437 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.578411 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 15:55:59.582555 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.579491 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 15:55:59.582555 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.580525 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 15:55:59.582555 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.580590 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rgvq9\"" Sep 10 15:55:59.582555 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.580847 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 15:55:59.582555 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.581231 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 15:55:59.582918 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.582888 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:55:59.582988 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.582966 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:55:59.586150 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.586061 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.588883 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.588781 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:55:59.588883 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.588850 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:55:59.590687 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.589986 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 15:55:59.590687 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.590237 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 15:55:59.590687 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.590485 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4v8sd\"" Sep 10 15:55:59.590687 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.590495 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 15:55:59.607201 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.607178 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tqr4q" Sep 10 15:55:59.614702 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.614682 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tqr4q" Sep 10 15:55:59.662235 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.662169 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.676964 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-log-socket\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677004 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2lnd\" (UniqueName: \"kubernetes.io/projected/0a39261f-d1f2-438d-a6ef-b83066bbead7-kube-api-access-m2lnd\") pod \"node-resolver-86g67\" (UID: \"0a39261f-d1f2-438d-a6ef-b83066bbead7\") " pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677038 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sksnp\" (UniqueName: \"kubernetes.io/projected/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-kube-api-access-sksnp\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677069 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677083 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-log-socket\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677094 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-run\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677147 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7438a602-18a4-48c3-aa99-7a3cdd98cae7-host\") pod \"node-ca-sms7n\" (UID: \"7438a602-18a4-48c3-aa99-7a3cdd98cae7\") " pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677179 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-kubelet\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677207 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-cni-binary-copy\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677233 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-run\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677233 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-var-lib-cni-multus\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677257 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-kubelet\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677267 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-conf-dir\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.677305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677312 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677341 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-run-systemd\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-run-ovn\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677391 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-node-log\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677415 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/954fee05-d00a-467e-b33b-3f16ded2d70a-agent-certs\") pod \"konnectivity-agent-w9mp9\" (UID: \"954fee05-d00a-467e-b33b-3f16ded2d70a\") " pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677441 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/466e913c-b578-40aa-b099-669bdfe2ecd5-kubelet-config\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677448 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-run-ovn\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677483 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-run-systemd\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677488 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-node-log\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677503 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/466e913c-b578-40aa-b099-669bdfe2ecd5-dbus\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677523 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-socket-dir-parent\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677542 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c29a7638-5037-4b0e-91f8-074b80b0bb61-tmp\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677557 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-var-lib-openvswitch\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677573 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-cni-netd\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677603 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-cni-netd\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677619 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-lib-modules\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677635 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-host\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.678082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677651 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677668 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-registration-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677683 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-cnibin\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677698 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677714 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-env-overrides\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677758 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-ovnkube-script-lib\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677777 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-modprobe-d\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677793 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677816 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-cnibin\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677838 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-run-k8s-cni-cncf-io\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677855 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-run-openvswitch\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677871 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a39261f-d1f2-438d-a6ef-b83066bbead7-tmp-dir\") pod \"node-resolver-86g67\" (UID: \"0a39261f-d1f2-438d-a6ef-b83066bbead7\") " pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677885 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677900 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-socket-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677915 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-daemon-config\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677930 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbdvk\" (UniqueName: \"kubernetes.io/projected/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-kube-api-access-sbdvk\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677947 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqw4j\" (UniqueName: \"kubernetes.io/projected/5c40402d-c9b9-4d90-9d8f-1af6003a16e5-kube-api-access-fqw4j\") pod \"iptables-alerter-6gf2s\" (UID: \"5c40402d-c9b9-4d90-9d8f-1af6003a16e5\") " pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.679030 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677962 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-sysctl-conf\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677978 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-systemd\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.677999 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678015 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-device-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678032 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-system-cni-dir\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678047 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-var-lib-kubelet\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678064 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-modprobe-d\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678073 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a39261f-d1f2-438d-a6ef-b83066bbead7-hosts-file\") pod \"node-resolver-86g67\" (UID: \"0a39261f-d1f2-438d-a6ef-b83066bbead7\") " pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678158 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7438a602-18a4-48c3-aa99-7a3cdd98cae7-serviceca\") pod \"node-ca-sms7n\" (UID: \"7438a602-18a4-48c3-aa99-7a3cdd98cae7\") " pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678208 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a39261f-d1f2-438d-a6ef-b83066bbead7-hosts-file\") pod \"node-resolver-86g67\" (UID: \"0a39261f-d1f2-438d-a6ef-b83066bbead7\") " pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678238 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-run-openvswitch\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678238 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-lib-modules\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678317 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-sysctl-conf\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678348 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-var-lib-openvswitch\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678382 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-host\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678426 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678454 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s25dj\" (UniqueName: \"kubernetes.io/projected/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-kube-api-access-s25dj\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.679965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678511 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678541 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-etc-selinux\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678542 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a39261f-d1f2-438d-a6ef-b83066bbead7-tmp-dir\") pod \"node-resolver-86g67\" (UID: \"0a39261f-d1f2-438d-a6ef-b83066bbead7\") " pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678595 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-systemd\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-sys-fs\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678632 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-hostroot\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678659 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678697 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5c40402d-c9b9-4d90-9d8f-1af6003a16e5-iptables-alerter-script\") pod \"iptables-alerter-6gf2s\" (UID: \"5c40402d-c9b9-4d90-9d8f-1af6003a16e5\") " pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678722 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-slash\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678752 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-etc-openvswitch\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678802 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-ovn-node-metrics-cert\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678856 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t2zn\" (UniqueName: \"kubernetes.io/projected/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-kube-api-access-5t2zn\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678862 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-ovnkube-script-lib\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678883 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-sys\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678913 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-sysctl-d\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678941 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/954fee05-d00a-467e-b33b-3f16ded2d70a-konnectivity-ca\") pod \"konnectivity-agent-w9mp9\" (UID: \"954fee05-d00a-467e-b33b-3f16ded2d70a\") " pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678967 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwds2\" (UniqueName: \"kubernetes.io/projected/c29a7638-5037-4b0e-91f8-074b80b0bb61-kube-api-access-hwds2\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.680925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.678992 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-run-netns\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679059 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-env-overrides\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679090 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679117 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-systemd-units\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679142 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-kubernetes\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679173 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-run-netns\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679199 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-var-lib-cni-bin\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679226 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-run-multus-certs\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679254 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgnm9\" (UniqueName: \"kubernetes.io/projected/43c09421-c94c-4ec6-abe4-a30be09975c6-kube-api-access-fgnm9\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679283 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-cni-bin\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679305 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-run-netns\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679353 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-var-lib-kubelet\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679382 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-system-cni-dir\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679437 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wht2q\" (UniqueName: \"kubernetes.io/projected/7438a602-18a4-48c3-aa99-7a3cdd98cae7-kube-api-access-wht2q\") pod \"node-ca-sms7n\" (UID: \"7438a602-18a4-48c3-aa99-7a3cdd98cae7\") " pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679482 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-etc-kubernetes\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679493 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679512 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c40402d-c9b9-4d90-9d8f-1af6003a16e5-host-slash\") pod \"iptables-alerter-6gf2s\" (UID: \"5c40402d-c9b9-4d90-9d8f-1af6003a16e5\") " pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.681834 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679540 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-ovnkube-config\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679618 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-sysconfig\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679647 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-cni-dir\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679677 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-cni-binary-copy\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-tuned\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679674 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-cni-bin\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679787 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-sys\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679843 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5c40402d-c9b9-4d90-9d8f-1af6003a16e5-iptables-alerter-script\") pod \"iptables-alerter-6gf2s\" (UID: \"5c40402d-c9b9-4d90-9d8f-1af6003a16e5\") " pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679893 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-sysctl-d\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679895 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-etc-openvswitch\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679925 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-host-slash\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.679997 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-kubernetes\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.680165 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-systemd-units\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.680225 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.680245 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-sysconfig\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.680296 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c40402d-c9b9-4d90-9d8f-1af6003a16e5-host-slash\") pod \"iptables-alerter-6gf2s\" (UID: \"5c40402d-c9b9-4d90-9d8f-1af6003a16e5\") " pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.680314 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs podName:43c09421-c94c-4ec6-abe4-a30be09975c6 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:00.180279913 +0000 UTC m=+2.956338351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs") pod "network-metrics-daemon-jclb4" (UID: "43c09421-c94c-4ec6-abe4-a30be09975c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:55:59.682748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.680396 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c29a7638-5037-4b0e-91f8-074b80b0bb61-var-lib-kubelet\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.683629 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.680430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-ovnkube-config\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.683825 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.683803 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c29a7638-5037-4b0e-91f8-074b80b0bb61-tmp\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.683962 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.683903 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-ovn-node-metrics-cert\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.684069 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.683994 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c29a7638-5037-4b0e-91f8-074b80b0bb61-etc-tuned\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.692013 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.691988 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t2zn\" (UniqueName: \"kubernetes.io/projected/5a42ae36-0842-4ae5-8d49-a00d603fdd5b-kube-api-access-5t2zn\") pod \"ovnkube-node-7zwpx\" (UID: \"5a42ae36-0842-4ae5-8d49-a00d603fdd5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.692123 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.692013 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqw4j\" (UniqueName: \"kubernetes.io/projected/5c40402d-c9b9-4d90-9d8f-1af6003a16e5-kube-api-access-fqw4j\") pod \"iptables-alerter-6gf2s\" (UID: \"5c40402d-c9b9-4d90-9d8f-1af6003a16e5\") " pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.693211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.693176 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwds2\" (UniqueName: \"kubernetes.io/projected/c29a7638-5037-4b0e-91f8-074b80b0bb61-kube-api-access-hwds2\") pod \"tuned-tdhmk\" (UID: \"c29a7638-5037-4b0e-91f8-074b80b0bb61\") " pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.693494 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.693466 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgnm9\" (UniqueName: \"kubernetes.io/projected/43c09421-c94c-4ec6-abe4-a30be09975c6-kube-api-access-fgnm9\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:55:59.693577 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.693476 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2lnd\" (UniqueName: \"kubernetes.io/projected/0a39261f-d1f2-438d-a6ef-b83066bbead7-kube-api-access-m2lnd\") pod \"node-resolver-86g67\" (UID: \"0a39261f-d1f2-438d-a6ef-b83066bbead7\") " pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.699589 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.699542 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" event={"ID":"ae9bd1aaa63cabc2deea36246571659b","Type":"ContainerStarted","Data":"80ad983c167c1796dfce76584c22ae28e189b573e27dad0af2289c2c5b15a311"} Sep 10 15:55:59.701832 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.701805 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal" event={"ID":"e59b8d96466349d07517a47588dc5f77","Type":"ContainerStarted","Data":"15672d867c627414c17ce5eb8aa77f07a0946bf6c6b1cca64b76c7e72ff7fd9c"} Sep 10 15:55:59.741854 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.741828 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 15:55:59.780169 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780137 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:55:59.780292 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780184 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-registration-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.780292 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-cnibin\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.780292 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780239 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:55:59.780292 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780264 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-cnibin\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.780292 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780289 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-run-k8s-cni-cncf-io\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780324 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-registration-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780339 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-socket-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780363 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-daemon-config\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.780367 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780378 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-cnibin\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780387 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbdvk\" (UniqueName: \"kubernetes.io/projected/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-kube-api-access-sbdvk\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780428 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-cnibin\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.780432 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret podName:466e913c-b578-40aa-b099-669bdfe2ecd5 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:00.280412618 +0000 UTC m=+3.056471046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret") pod "global-pull-secret-syncer-q49dm" (UID: "466e913c-b578-40aa-b099-669bdfe2ecd5") : object "kube-system"/"original-pull-secret" not registered Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780480 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-device-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780484 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-run-k8s-cni-cncf-io\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780505 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-system-cni-dir\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780529 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-var-lib-kubelet\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780553 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7438a602-18a4-48c3-aa99-7a3cdd98cae7-serviceca\") pod \"node-ca-sms7n\" (UID: \"7438a602-18a4-48c3-aa99-7a3cdd98cae7\") " pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.780570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780576 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s25dj\" (UniqueName: \"kubernetes.io/projected/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-kube-api-access-s25dj\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-etc-selinux\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780647 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-sys-fs\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780669 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-hostroot\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780693 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780718 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-socket-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780728 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/954fee05-d00a-467e-b33b-3f16ded2d70a-konnectivity-ca\") pod \"konnectivity-agent-w9mp9\" (UID: \"954fee05-d00a-467e-b33b-3f16ded2d70a\") " pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780804 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-run-netns\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780832 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-var-lib-cni-bin\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780863 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-run-multus-certs\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780895 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-system-cni-dir\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780919 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wht2q\" (UniqueName: \"kubernetes.io/projected/7438a602-18a4-48c3-aa99-7a3cdd98cae7-kube-api-access-wht2q\") pod \"node-ca-sms7n\" (UID: \"7438a602-18a4-48c3-aa99-7a3cdd98cae7\") " pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780945 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-etc-kubernetes\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.780976 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-cni-dir\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781000 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-cni-binary-copy\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781031 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sksnp\" (UniqueName: \"kubernetes.io/projected/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-kube-api-access-sksnp\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.781483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781061 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7438a602-18a4-48c3-aa99-7a3cdd98cae7-host\") pod \"node-ca-sms7n\" (UID: \"7438a602-18a4-48c3-aa99-7a3cdd98cae7\") " pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781092 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-cni-binary-copy\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781117 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-var-lib-cni-multus\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781142 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-conf-dir\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781175 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/954fee05-d00a-467e-b33b-3f16ded2d70a-agent-certs\") pod \"konnectivity-agent-w9mp9\" (UID: \"954fee05-d00a-467e-b33b-3f16ded2d70a\") " pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781200 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/466e913c-b578-40aa-b099-669bdfe2ecd5-kubelet-config\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781225 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/466e913c-b578-40aa-b099-669bdfe2ecd5-dbus\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781252 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-socket-dir-parent\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781283 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-daemon-config\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-system-cni-dir\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781354 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-device-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781356 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-socket-dir-parent\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781384 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-var-lib-kubelet\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781496 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781552 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-run-netns\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781283 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/954fee05-d00a-467e-b33b-3f16ded2d70a-konnectivity-ca\") pod \"konnectivity-agent-w9mp9\" (UID: \"954fee05-d00a-467e-b33b-3f16ded2d70a\") " pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781702 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-etc-selinux\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781828 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7438a602-18a4-48c3-aa99-7a3cdd98cae7-serviceca\") pod \"node-ca-sms7n\" (UID: \"7438a602-18a4-48c3-aa99-7a3cdd98cae7\") " pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.782408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781887 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-sys-fs\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781928 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-hostroot\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.781964 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782034 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7438a602-18a4-48c3-aa99-7a3cdd98cae7-host\") pod \"node-ca-sms7n\" (UID: \"7438a602-18a4-48c3-aa99-7a3cdd98cae7\") " pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782081 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-run-multus-certs\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782113 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782125 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-system-cni-dir\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782388 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-etc-kubernetes\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782418 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782487 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/466e913c-b578-40aa-b099-669bdfe2ecd5-kubelet-config\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782529 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-cni-dir\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782532 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-var-lib-cni-multus\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782568 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-multus-conf-dir\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782578 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-cni-binary-copy\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782662 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/466e913c-b578-40aa-b099-669bdfe2ecd5-dbus\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.782711 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-host-var-lib-cni-bin\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.783338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.783037 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-cni-binary-copy\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.786154 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.786130 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/954fee05-d00a-467e-b33b-3f16ded2d70a-agent-certs\") pod \"konnectivity-agent-w9mp9\" (UID: \"954fee05-d00a-467e-b33b-3f16ded2d70a\") " pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:55:59.794722 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.794654 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 15:55:59.794722 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.794677 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 15:55:59.794722 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.794691 2576 projected.go:196] Error preparing data for projected volume kube-api-access-5qh7f for pod openshift-network-diagnostics/network-check-target-mwvrc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:55:59.794925 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:55:59.794755 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f podName:67089a15-730b-4bc3-84ad-a429e8edc246 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:00.29473961 +0000 UTC m=+3.070798024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5qh7f" (UniqueName: "kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f") pod "network-check-target-mwvrc" (UID: "67089a15-730b-4bc3-84ad-a429e8edc246") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:55:59.805221 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.805175 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s25dj\" (UniqueName: \"kubernetes.io/projected/e92b2fbf-b2f0-476b-842a-93d60ce94b8c-kube-api-access-s25dj\") pod \"multus-additional-cni-plugins-nq9fq\" (UID: \"e92b2fbf-b2f0-476b-842a-93d60ce94b8c\") " pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.806060 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.806016 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sksnp\" (UniqueName: \"kubernetes.io/projected/44dde14f-f0aa-4b2b-a599-f51fba41e0b1-kube-api-access-sksnp\") pod \"aws-ebs-csi-driver-node-btbzn\" (UID: \"44dde14f-f0aa-4b2b-a599-f51fba41e0b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.806741 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.806697 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wht2q\" (UniqueName: \"kubernetes.io/projected/7438a602-18a4-48c3-aa99-7a3cdd98cae7-kube-api-access-wht2q\") pod \"node-ca-sms7n\" (UID: \"7438a602-18a4-48c3-aa99-7a3cdd98cae7\") " pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:55:59.807687 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.807644 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbdvk\" (UniqueName: \"kubernetes.io/projected/b03fdcb9-b788-4551-8f73-b5cce9a1e52c-kube-api-access-sbdvk\") pod \"multus-4kfq7\" (UID: \"b03fdcb9-b788-4551-8f73-b5cce9a1e52c\") " pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.872965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.872934 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-86g67" Sep 10 15:55:59.887942 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.887922 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6gf2s" Sep 10 15:55:59.898121 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.898085 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:55:59.904189 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.904167 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" Sep 10 15:55:59.913064 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.913038 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" Sep 10 15:55:59.922110 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.921947 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4kfq7" Sep 10 15:55:59.930081 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.930043 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:55:59.939821 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.939795 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" Sep 10 15:55:59.947755 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:55:59.947735 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sms7n" Sep 10 15:56:00.184071 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:00.183980 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:00.184242 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:00.184109 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:00.184242 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:00.184167 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs podName:43c09421-c94c-4ec6-abe4-a30be09975c6 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:01.184146524 +0000 UTC m=+3.960204940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs") pod "network-metrics-daemon-jclb4" (UID: "43c09421-c94c-4ec6-abe4-a30be09975c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:00.285237 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:00.285204 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:00.285407 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:00.285383 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:00.285473 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:00.285461 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret podName:466e913c-b578-40aa-b099-669bdfe2ecd5 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:01.285438576 +0000 UTC m=+4.061496995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret") pod "global-pull-secret-syncer-q49dm" (UID: "466e913c-b578-40aa-b099-669bdfe2ecd5") : object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:00.385597 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:00.385561 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:00.385806 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:00.385739 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 15:56:00.385806 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:00.385767 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 15:56:00.385806 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:00.385779 2576 projected.go:196] Error preparing data for projected volume kube-api-access-5qh7f for pod openshift-network-diagnostics/network-check-target-mwvrc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:00.385967 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:00.385850 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f podName:67089a15-730b-4bc3-84ad-a429e8edc246 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:01.385828933 +0000 UTC m=+4.161887344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qh7f" (UniqueName: "kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f") pod "network-check-target-mwvrc" (UID: "67089a15-730b-4bc3-84ad-a429e8edc246") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:00.578984 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:56:00.578912 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a39261f_d1f2_438d_a6ef_b83066bbead7.slice/crio-deec08e35fecb0a7995686e366276e9c68b8baa327a27cc16e70c9738b8f0042 WatchSource:0}: Error finding container deec08e35fecb0a7995686e366276e9c68b8baa327a27cc16e70c9738b8f0042: Status 404 returned error can't find the container with id deec08e35fecb0a7995686e366276e9c68b8baa327a27cc16e70c9738b8f0042 Sep 10 15:56:00.618458 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:00.618422 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:51:00 +0000 UTC" deadline="2028-03-20 18:56:44.932524047 +0000 UTC" Sep 10 15:56:00.618458 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:00.618456 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13371h0m44.314072852s" Sep 10 15:56:00.694350 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:00.694315 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:00.694510 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:00.694466 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:00.704775 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:00.704741 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-86g67" event={"ID":"0a39261f-d1f2-438d-a6ef-b83066bbead7","Type":"ContainerStarted","Data":"deec08e35fecb0a7995686e366276e9c68b8baa327a27cc16e70c9738b8f0042"} Sep 10 15:56:00.959990 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:56:00.959955 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92b2fbf_b2f0_476b_842a_93d60ce94b8c.slice/crio-434c6d0844a8f5902c9420008b3eb0d310401476e6b5b31cdfefa17315ae0e2e WatchSource:0}: Error finding container 434c6d0844a8f5902c9420008b3eb0d310401476e6b5b31cdfefa17315ae0e2e: Status 404 returned error can't find the container with id 434c6d0844a8f5902c9420008b3eb0d310401476e6b5b31cdfefa17315ae0e2e Sep 10 15:56:00.961842 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:56:00.960853 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7438a602_18a4_48c3_aa99_7a3cdd98cae7.slice/crio-f5446678ef28e6082589fd5babfdda45df0ee567fe13872071f4d71a79b7fccd WatchSource:0}: Error finding container f5446678ef28e6082589fd5babfdda45df0ee567fe13872071f4d71a79b7fccd: Status 404 returned error can't find the container with id f5446678ef28e6082589fd5babfdda45df0ee567fe13872071f4d71a79b7fccd Sep 10 15:56:00.962671 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:56:00.962636 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb03fdcb9_b788_4551_8f73_b5cce9a1e52c.slice/crio-c87c83b81aa60a07bb088326c6dba7687441d3c29b1a1edbdf42ea20e604a662 WatchSource:0}: Error finding container c87c83b81aa60a07bb088326c6dba7687441d3c29b1a1edbdf42ea20e604a662: Status 404 returned error can't find the container with id c87c83b81aa60a07bb088326c6dba7687441d3c29b1a1edbdf42ea20e604a662 Sep 10 15:56:00.965133 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:56:00.964938 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29a7638_5037_4b0e_91f8_074b80b0bb61.slice/crio-3bbbb85fa66bfb1e44c4e8cce41229534e2c427dcc325b9e3dfa5e19e007464b WatchSource:0}: Error finding container 3bbbb85fa66bfb1e44c4e8cce41229534e2c427dcc325b9e3dfa5e19e007464b: Status 404 returned error can't find the container with id 3bbbb85fa66bfb1e44c4e8cce41229534e2c427dcc325b9e3dfa5e19e007464b Sep 10 15:56:00.989338 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:56:00.989311 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c40402d_c9b9_4d90_9d8f_1af6003a16e5.slice/crio-79c1f2e1e00f418cd23d051e58dd5a3cd5694bdf94958418e8739aacd266ee4c WatchSource:0}: Error finding container 79c1f2e1e00f418cd23d051e58dd5a3cd5694bdf94958418e8739aacd266ee4c: Status 404 returned error can't find the container with id 79c1f2e1e00f418cd23d051e58dd5a3cd5694bdf94958418e8739aacd266ee4c Sep 10 15:56:00.990054 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:56:00.990032 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44dde14f_f0aa_4b2b_a599_f51fba41e0b1.slice/crio-c4f67e90782fbf3917b4a5e074cfbd7c74c720797303b99080ee314ccb6dad8c WatchSource:0}: Error finding container c4f67e90782fbf3917b4a5e074cfbd7c74c720797303b99080ee314ccb6dad8c: Status 404 returned error can't find the container with id c4f67e90782fbf3917b4a5e074cfbd7c74c720797303b99080ee314ccb6dad8c Sep 10 15:56:00.991058 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:56:00.990979 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a42ae36_0842_4ae5_8d49_a00d603fdd5b.slice/crio-e9f7a4bbb55f705fcaad7bffbe9759fd616ef4076aabd36342338e71ee74f3e9 WatchSource:0}: Error finding container e9f7a4bbb55f705fcaad7bffbe9759fd616ef4076aabd36342338e71ee74f3e9: Status 404 returned error can't find the container with id e9f7a4bbb55f705fcaad7bffbe9759fd616ef4076aabd36342338e71ee74f3e9 Sep 10 15:56:00.992510 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:56:00.992486 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod954fee05_d00a_467e_b33b_3f16ded2d70a.slice/crio-7594d41e4a2d40334db4e7c4c7beff4a1ac9f66e7f484462237b0a5435133cbb WatchSource:0}: Error finding container 7594d41e4a2d40334db4e7c4c7beff4a1ac9f66e7f484462237b0a5435133cbb: Status 404 returned error can't find the container with id 7594d41e4a2d40334db4e7c4c7beff4a1ac9f66e7f484462237b0a5435133cbb Sep 10 15:56:00.994632 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:00.994610 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 15:56:01.191469 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.191409 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:01.191645 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.191615 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:01.191714 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.191707 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs podName:43c09421-c94c-4ec6-abe4-a30be09975c6 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:03.191682916 +0000 UTC m=+5.967741349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs") pod "network-metrics-daemon-jclb4" (UID: "43c09421-c94c-4ec6-abe4-a30be09975c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:01.292057 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.292028 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:01.292210 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.292190 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:01.292282 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.292265 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret podName:466e913c-b578-40aa-b099-669bdfe2ecd5 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:03.292243778 +0000 UTC m=+6.068302204 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret") pod "global-pull-secret-syncer-q49dm" (UID: "466e913c-b578-40aa-b099-669bdfe2ecd5") : object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:01.393166 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.393131 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:01.393325 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.393308 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 15:56:01.393389 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.393330 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 15:56:01.393389 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.393343 2576 projected.go:196] Error preparing data for projected volume kube-api-access-5qh7f for pod openshift-network-diagnostics/network-check-target-mwvrc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:01.393454 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.393409 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f podName:67089a15-730b-4bc3-84ad-a429e8edc246 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:03.393392133 +0000 UTC m=+6.169450557 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qh7f" (UniqueName: "kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f") pod "network-check-target-mwvrc" (UID: "67089a15-730b-4bc3-84ad-a429e8edc246") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:01.619242 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.619115 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:51:00 +0000 UTC" deadline="2028-05-09 09:12:58.976344733 +0000 UTC" Sep 10 15:56:01.619242 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.619173 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14561h16m57.357175775s" Sep 10 15:56:01.697770 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.697179 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:01.697770 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.697312 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:01.698662 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.698508 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:01.698662 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:01.698611 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:01.719635 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.719603 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerStarted","Data":"e9f7a4bbb55f705fcaad7bffbe9759fd616ef4076aabd36342338e71ee74f3e9"} Sep 10 15:56:01.723595 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.723556 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w9mp9" event={"ID":"954fee05-d00a-467e-b33b-3f16ded2d70a","Type":"ContainerStarted","Data":"7594d41e4a2d40334db4e7c4c7beff4a1ac9f66e7f484462237b0a5435133cbb"} Sep 10 15:56:01.728847 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.728788 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" event={"ID":"44dde14f-f0aa-4b2b-a599-f51fba41e0b1","Type":"ContainerStarted","Data":"c4f67e90782fbf3917b4a5e074cfbd7c74c720797303b99080ee314ccb6dad8c"} Sep 10 15:56:01.731985 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.731924 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" event={"ID":"c29a7638-5037-4b0e-91f8-074b80b0bb61","Type":"ContainerStarted","Data":"3bbbb85fa66bfb1e44c4e8cce41229534e2c427dcc325b9e3dfa5e19e007464b"} Sep 10 15:56:01.735223 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.735161 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sms7n" event={"ID":"7438a602-18a4-48c3-aa99-7a3cdd98cae7","Type":"ContainerStarted","Data":"f5446678ef28e6082589fd5babfdda45df0ee567fe13872071f4d71a79b7fccd"} Sep 10 15:56:01.737560 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.737492 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" event={"ID":"e92b2fbf-b2f0-476b-842a-93d60ce94b8c","Type":"ContainerStarted","Data":"434c6d0844a8f5902c9420008b3eb0d310401476e6b5b31cdfefa17315ae0e2e"} Sep 10 15:56:01.743813 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.742853 2576 generic.go:356] "Generic (PLEG): container finished" podID="ae9bd1aaa63cabc2deea36246571659b" containerID="763b892140c1518ede1b19d309e788613df1036ce0838edcb2759a28d6daf065" exitCode=0 Sep 10 15:56:01.743813 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.742933 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" event={"ID":"ae9bd1aaa63cabc2deea36246571659b","Type":"ContainerDied","Data":"763b892140c1518ede1b19d309e788613df1036ce0838edcb2759a28d6daf065"} Sep 10 15:56:01.748992 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.747597 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal" event={"ID":"e59b8d96466349d07517a47588dc5f77","Type":"ContainerStarted","Data":"31a6ab625c80b9eff8f89aea36908e861a38aee20727f47bd21dc86732b25e50"} Sep 10 15:56:01.752573 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.752525 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6gf2s" event={"ID":"5c40402d-c9b9-4d90-9d8f-1af6003a16e5","Type":"ContainerStarted","Data":"79c1f2e1e00f418cd23d051e58dd5a3cd5694bdf94958418e8739aacd266ee4c"} Sep 10 15:56:01.767220 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.765385 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4kfq7" event={"ID":"b03fdcb9-b788-4551-8f73-b5cce9a1e52c","Type":"ContainerStarted","Data":"c87c83b81aa60a07bb088326c6dba7687441d3c29b1a1edbdf42ea20e604a662"} Sep 10 15:56:01.787071 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:01.786909 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-74.ec2.internal" podStartSLOduration=2.786890073 podStartE2EDuration="2.786890073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:01.786014363 +0000 UTC m=+4.562072800" watchObservedRunningTime="2026-09-10 15:56:01.786890073 +0000 UTC m=+4.562948510" Sep 10 15:56:02.694597 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:02.694292 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:02.694597 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:02.694449 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:02.787288 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:02.786849 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" event={"ID":"ae9bd1aaa63cabc2deea36246571659b","Type":"ContainerStarted","Data":"02345de8553c4c8544eccb8f1736ecfe2748b0220b9216f056ee8d8bc0ce6cff"} Sep 10 15:56:02.838745 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:02.838683 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-74.ec2.internal" podStartSLOduration=3.838660367 podStartE2EDuration="3.838660367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:02.837656367 +0000 UTC m=+5.613714806" watchObservedRunningTime="2026-09-10 15:56:02.838660367 +0000 UTC m=+5.614718805" Sep 10 15:56:03.211188 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:03.211126 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:03.211382 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.211279 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:03.211382 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.211340 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs podName:43c09421-c94c-4ec6-abe4-a30be09975c6 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:07.211321648 +0000 UTC m=+9.987380065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs") pod "network-metrics-daemon-jclb4" (UID: "43c09421-c94c-4ec6-abe4-a30be09975c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:03.311900 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:03.311862 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:03.312077 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.312038 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:03.312158 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.312103 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret podName:466e913c-b578-40aa-b099-669bdfe2ecd5 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:07.312084253 +0000 UTC m=+10.088142676 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret") pod "global-pull-secret-syncer-q49dm" (UID: "466e913c-b578-40aa-b099-669bdfe2ecd5") : object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:03.412807 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:03.412773 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:03.412995 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.412971 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 15:56:03.412995 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.412994 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 15:56:03.413099 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.413008 2576 projected.go:196] Error preparing data for projected volume kube-api-access-5qh7f for pod openshift-network-diagnostics/network-check-target-mwvrc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:03.413099 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.413065 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f podName:67089a15-730b-4bc3-84ad-a429e8edc246 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:07.413046263 +0000 UTC m=+10.189104677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qh7f" (UniqueName: "kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f") pod "network-check-target-mwvrc" (UID: "67089a15-730b-4bc3-84ad-a429e8edc246") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:03.696838 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:03.696748 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:03.697316 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.696933 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:03.697316 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:03.697067 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:03.697316 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:03.697147 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:04.694409 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:04.694372 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:04.694617 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:04.694524 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:05.694474 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:05.693833 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:05.694474 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:05.693963 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:05.694474 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:05.694030 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:05.694474 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:05.694105 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:06.694379 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:06.694298 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:06.694542 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:06.694422 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:07.250570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:07.250535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:07.250820 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.250679 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:07.250820 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.250759 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs podName:43c09421-c94c-4ec6-abe4-a30be09975c6 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:15.250739847 +0000 UTC m=+18.026798263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs") pod "network-metrics-daemon-jclb4" (UID: "43c09421-c94c-4ec6-abe4-a30be09975c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:07.351285 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:07.351247 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:07.351439 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.351388 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:07.351439 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.351438 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret podName:466e913c-b578-40aa-b099-669bdfe2ecd5 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:15.351423897 +0000 UTC m=+18.127482307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret") pod "global-pull-secret-syncer-q49dm" (UID: "466e913c-b578-40aa-b099-669bdfe2ecd5") : object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:07.452119 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:07.452080 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:07.452285 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.452224 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 15:56:07.452285 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.452246 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 15:56:07.452285 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.452260 2576 projected.go:196] Error preparing data for projected volume kube-api-access-5qh7f for pod openshift-network-diagnostics/network-check-target-mwvrc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:07.452444 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.452321 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f podName:67089a15-730b-4bc3-84ad-a429e8edc246 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:15.452302875 +0000 UTC m=+18.228361308 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qh7f" (UniqueName: "kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f") pod "network-check-target-mwvrc" (UID: "67089a15-730b-4bc3-84ad-a429e8edc246") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:07.694857 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:07.694784 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:07.695299 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.694898 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:07.695299 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:07.695245 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:07.695403 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:07.695336 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:08.694048 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:08.693687 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:08.694048 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:08.693848 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:09.694541 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:09.694462 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:09.694931 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:09.694589 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:09.695051 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:09.695020 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:09.695182 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:09.695116 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:10.693516 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:10.693487 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:10.693657 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:10.693595 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:11.694126 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:11.694095 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:11.694553 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:11.694212 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:11.694553 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:11.694267 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:11.694553 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:11.694383 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:12.693518 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:12.693483 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:12.693701 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:12.693626 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:13.694045 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:13.694002 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:13.694469 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:13.694002 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:13.694469 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:13.694147 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:13.694469 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:13.694254 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:14.693882 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:14.693852 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:14.694093 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:14.693969 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:15.309767 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:15.309729 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:15.309957 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.309889 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:15.310016 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.309967 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs podName:43c09421-c94c-4ec6-abe4-a30be09975c6 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:31.30994533 +0000 UTC m=+34.086003743 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs") pod "network-metrics-daemon-jclb4" (UID: "43c09421-c94c-4ec6-abe4-a30be09975c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:15.410347 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:15.410308 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:15.410519 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.410473 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:15.410568 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.410536 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret podName:466e913c-b578-40aa-b099-669bdfe2ecd5 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:31.41052191 +0000 UTC m=+34.186580326 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret") pod "global-pull-secret-syncer-q49dm" (UID: "466e913c-b578-40aa-b099-669bdfe2ecd5") : object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:15.511215 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:15.511180 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:15.511378 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.511356 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 15:56:15.511455 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.511385 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 15:56:15.511455 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.511398 2576 projected.go:196] Error preparing data for projected volume kube-api-access-5qh7f for pod openshift-network-diagnostics/network-check-target-mwvrc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:15.511551 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.511459 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f podName:67089a15-730b-4bc3-84ad-a429e8edc246 nodeName:}" failed. No retries permitted until 2026-09-10 15:56:31.511440053 +0000 UTC m=+34.287498472 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qh7f" (UniqueName: "kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f") pod "network-check-target-mwvrc" (UID: "67089a15-730b-4bc3-84ad-a429e8edc246") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:15.694000 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:15.693918 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:15.694157 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:15.693933 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:15.694157 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.694057 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:15.694157 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:15.694093 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:16.694119 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:16.694082 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:16.694296 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:16.694196 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:17.694928 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:17.694895 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:17.695363 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:17.694981 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:17.695363 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:17.695000 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:17.695363 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:17.695096 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:18.693601 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:18.693572 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:18.693751 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:18.693669 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:19.693844 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.693525 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:19.694824 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.693585 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:19.694824 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:19.693912 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:19.694824 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:19.693977 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:19.840748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.840611 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-86g67" event={"ID":"0a39261f-d1f2-438d-a6ef-b83066bbead7","Type":"ContainerStarted","Data":"d222f33e7abec94d306688d0999be2f682f63d31d2211d02526bab5494384c63"} Sep 10 15:56:19.854379 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.854359 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 15:56:19.854784 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.854762 2576 generic.go:356] "Generic (PLEG): container finished" podID="5a42ae36-0842-4ae5-8d49-a00d603fdd5b" containerID="e7de43613ccc2c3e464ec55b2466a486ba5e86da93a9045dab80030b702dafb0" exitCode=1 Sep 10 15:56:19.854873 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.854835 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerStarted","Data":"fb06155b611b8854abcc553573400067f835680ca3f8af823fd438b498742885"} Sep 10 15:56:19.854938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.854895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerStarted","Data":"1441c1e7f227e543f7b419ec8fd737ed785253548841894b835ad04a305cea7d"} Sep 10 15:56:19.854938 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.854935 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerStarted","Data":"6e50c918b1a5895fe143b3c7836d095d9ca52e462681ef63bdfcd5d04ae5129e"} Sep 10 15:56:19.855036 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.854965 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerStarted","Data":"bd008c9e8c5501e2467c87f2051846e7747f85daf3f0ef0d5b294f1d9a0626d1"} Sep 10 15:56:19.855036 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.854997 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerDied","Data":"e7de43613ccc2c3e464ec55b2466a486ba5e86da93a9045dab80030b702dafb0"} Sep 10 15:56:19.855036 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.855031 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerStarted","Data":"73eadca54b1aa40ae9c70bc362aeaf87eed411777e828799a84c92e3e9d39614"} Sep 10 15:56:19.856431 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.856410 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w9mp9" event={"ID":"954fee05-d00a-467e-b33b-3f16ded2d70a","Type":"ContainerStarted","Data":"2d2f8b603bfc37747697f73c63a1edd1ac280aaa7ab15119f47531ee8246eb4f"} Sep 10 15:56:19.858074 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.858052 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" event={"ID":"44dde14f-f0aa-4b2b-a599-f51fba41e0b1","Type":"ContainerStarted","Data":"0b977640f8c7db4f828e957505ec837409a59383c5a0b08a8ea146fa22ec970f"} Sep 10 15:56:19.859512 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.859475 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-86g67" podStartSLOduration=7.661054569 podStartE2EDuration="21.859463633s" totalImagesPullingTime="14.198409064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:19.859013112 +0000 UTC m=+22.635071543" watchObservedRunningTime="2026-09-10 15:56:19.859463633 +0000 UTC m=+22.635522065" Sep 10 15:56:19.859670 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.859652 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" event={"ID":"c29a7638-5037-4b0e-91f8-074b80b0bb61","Type":"ContainerStarted","Data":"a94f3f7d40e489540aba186f6065a26ea2822e39b22d02ae4e372778de20d0db"} Sep 10 15:56:19.861043 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.861026 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sms7n" event={"ID":"7438a602-18a4-48c3-aa99-7a3cdd98cae7","Type":"ContainerStarted","Data":"9b1eb500b8e3d92d18d2f2ba85341158aa08f429b8ef706775b42cd23ac7bf9b"} Sep 10 15:56:19.862512 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.862495 2576 generic.go:356] "Generic (PLEG): container finished" podID="e92b2fbf-b2f0-476b-842a-93d60ce94b8c" containerID="9c58e14407ffc855704438880b21b84d653f63fb9386ed0c58d7190b894a990c" exitCode=0 Sep 10 15:56:19.862595 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.862556 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" event={"ID":"e92b2fbf-b2f0-476b-842a-93d60ce94b8c","Type":"ContainerDied","Data":"9c58e14407ffc855704438880b21b84d653f63fb9386ed0c58d7190b894a990c"} Sep 10 15:56:19.863929 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.863912 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4kfq7" event={"ID":"b03fdcb9-b788-4551-8f73-b5cce9a1e52c","Type":"ContainerStarted","Data":"d118ce4991790b5824b374cc354fc339978e12948aee793285f26a190a31416f"} Sep 10 15:56:19.874472 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.874441 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w9mp9" podStartSLOduration=3.988948024 podStartE2EDuration="21.874429657s" totalImagesPullingTime="17.885481633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:19.873941868 +0000 UTC m=+22.650000302" watchObservedRunningTime="2026-09-10 15:56:19.874429657 +0000 UTC m=+22.650488118" Sep 10 15:56:19.888467 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.888437 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sms7n" podStartSLOduration=3.970869823 podStartE2EDuration="21.888426273s" totalImagesPullingTime="17.91755645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:19.888314777 +0000 UTC m=+22.664373208" watchObservedRunningTime="2026-09-10 15:56:19.888426273 +0000 UTC m=+22.664484706" Sep 10 15:56:19.960402 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.960345 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4kfq7" podStartSLOduration=3.737517891 podStartE2EDuration="21.960321089s" totalImagesPullingTime="18.222803198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:19.959927037 +0000 UTC m=+22.735985496" watchObservedRunningTime="2026-09-10 15:56:19.960321089 +0000 UTC m=+22.736379520" Sep 10 15:56:19.960645 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:19.960598 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tdhmk" podStartSLOduration=4.065690761 podStartE2EDuration="21.960588028s" totalImagesPullingTime="17.894897267s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:19.936022101 +0000 UTC m=+22.712080532" watchObservedRunningTime="2026-09-10 15:56:19.960588028 +0000 UTC m=+22.736646505" Sep 10 15:56:20.183398 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.183373 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 15:56:20.203864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.203840 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:56:20.204280 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.204267 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:56:20.628379 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.628273 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T15:56:20.183390232Z","UUID":"7a347dea-6e6b-414d-839f-887f677591e9","Handler":null,"Name":"","Endpoint":""} Sep 10 15:56:20.630990 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.630966 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 15:56:20.631124 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.631002 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 15:56:20.694097 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.694069 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:20.694515 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:20.694211 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:20.869363 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.869327 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6gf2s" event={"ID":"5c40402d-c9b9-4d90-9d8f-1af6003a16e5","Type":"ContainerStarted","Data":"4e4647ceda28bf0c04f0fd551cc0e799c06540d7510db98ce751123b368ea371"} Sep 10 15:56:20.873541 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.873517 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" event={"ID":"44dde14f-f0aa-4b2b-a599-f51fba41e0b1","Type":"ContainerStarted","Data":"214c62a638cb26f5d88ce52d9cc2b50e68ac75fe8bb40bdcc8fc0bbc4896342f"} Sep 10 15:56:20.873913 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.873893 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:56:20.874685 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.874664 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w9mp9" Sep 10 15:56:20.901423 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:20.901334 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6gf2s" podStartSLOduration=9.093924935 podStartE2EDuration="22.901314407s" totalImagesPullingTime="13.807389472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:20.885241689 +0000 UTC m=+23.661300121" watchObservedRunningTime="2026-09-10 15:56:20.901314407 +0000 UTC m=+23.677372841" Sep 10 15:56:21.694103 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:21.694074 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:21.694533 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:21.694074 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:21.694533 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:21.694195 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:21.694533 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:21.694237 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:21.878586 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:21.878502 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 15:56:21.878980 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:21.878944 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerStarted","Data":"283e105a6a3fc683f41043df4aa0eb10c55e8ceade6168c068bd19797c08ed6e"} Sep 10 15:56:21.881283 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:21.881255 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" event={"ID":"44dde14f-f0aa-4b2b-a599-f51fba41e0b1","Type":"ContainerStarted","Data":"ddbd774661a321301b638c7fcc262d34688ed978cd3007fbd9d84ffb82a68c68"} Sep 10 15:56:21.902509 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:21.902457 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-btbzn" podStartSLOduration=4.117526678 podStartE2EDuration="23.90243819s" totalImagesPullingTime="19.784911512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:21.901862685 +0000 UTC m=+24.677921118" watchObservedRunningTime="2026-09-10 15:56:21.90243819 +0000 UTC m=+24.678496623" Sep 10 15:56:22.693549 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:22.693509 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:22.693725 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:22.693660 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:23.694453 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:23.694419 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:23.694880 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:23.694419 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:23.694880 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:23.694552 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:23.694880 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:23.694632 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:24.694322 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.694100 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:24.694451 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:24.694371 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:24.890197 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.890161 2576 generic.go:356] "Generic (PLEG): container finished" podID="e92b2fbf-b2f0-476b-842a-93d60ce94b8c" containerID="fd4cc7a2651de0e653baf24bfce81b27c682805389feb012ae84f36a1a446580" exitCode=0 Sep 10 15:56:24.891033 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.890241 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" event={"ID":"e92b2fbf-b2f0-476b-842a-93d60ce94b8c","Type":"ContainerDied","Data":"fd4cc7a2651de0e653baf24bfce81b27c682805389feb012ae84f36a1a446580"} Sep 10 15:56:24.893926 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.893912 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 15:56:24.894258 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.894241 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerStarted","Data":"2867ce59136c62a5e46d29ea56f00ad4372736780e3da0e1543845ccf06dd8b8"} Sep 10 15:56:24.894494 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.894472 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:56:24.894582 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.894525 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:56:24.894582 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.894562 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:56:24.894740 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.894595 2576 scope.go:117] "RemoveContainer" containerID="e7de43613ccc2c3e464ec55b2466a486ba5e86da93a9045dab80030b702dafb0" Sep 10 15:56:24.911254 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.911220 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:56:24.911337 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:24.911328 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:56:25.694486 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:25.694260 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:25.694583 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:25.694281 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:25.694652 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:25.694584 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:25.694720 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:25.694640 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:25.900488 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:25.900470 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 15:56:25.900870 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:25.900850 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" event={"ID":"5a42ae36-0842-4ae5-8d49-a00d603fdd5b","Type":"ContainerStarted","Data":"989025e79e1dea9c4acbc27544ceb5202d8d3b0995cacd71baae6aae1d010464"} Sep 10 15:56:25.902945 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:25.902920 2576 generic.go:356] "Generic (PLEG): container finished" podID="e92b2fbf-b2f0-476b-842a-93d60ce94b8c" containerID="96cb2d4731d248a1d239ca11fac0f15a2aa1a69e13d26cdebadf875283e70b15" exitCode=0 Sep 10 15:56:25.903048 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:25.902954 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" event={"ID":"e92b2fbf-b2f0-476b-842a-93d60ce94b8c","Type":"ContainerDied","Data":"96cb2d4731d248a1d239ca11fac0f15a2aa1a69e13d26cdebadf875283e70b15"} Sep 10 15:56:25.930062 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:25.929967 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" podStartSLOduration=10.013751265 podStartE2EDuration="27.929953372s" totalImagesPullingTime="17.916202107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:25.928731642 +0000 UTC m=+28.704790073" watchObservedRunningTime="2026-09-10 15:56:25.929953372 +0000 UTC m=+28.706011782" Sep 10 15:56:26.058103 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:26.058072 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mwvrc"] Sep 10 15:56:26.058285 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:26.058164 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:26.058285 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:26.058257 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:26.058981 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:26.058958 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jclb4"] Sep 10 15:56:26.059082 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:26.059068 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:26.059239 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:26.059219 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:26.060491 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:26.060469 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q49dm"] Sep 10 15:56:26.060568 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:26.060551 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:26.060666 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:26.060646 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:26.910226 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:26.910145 2576 generic.go:356] "Generic (PLEG): container finished" podID="e92b2fbf-b2f0-476b-842a-93d60ce94b8c" containerID="7bb3997cddd62afa5f3edab46f277935b4508fa1d7d37b8ef9be13cbd25c2b85" exitCode=0 Sep 10 15:56:26.910629 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:26.910233 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" event={"ID":"e92b2fbf-b2f0-476b-842a-93d60ce94b8c","Type":"ContainerDied","Data":"7bb3997cddd62afa5f3edab46f277935b4508fa1d7d37b8ef9be13cbd25c2b85"} Sep 10 15:56:27.695303 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:27.695280 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:27.695422 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:27.695357 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:27.695422 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:27.695375 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:27.695525 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:27.695454 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:27.695525 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:27.695503 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:27.695630 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:27.695559 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:29.075169 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:29.074967 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-86g67_0a39261f-d1f2-438d-a6ef-b83066bbead7/dns-node-resolver/0.log" Sep 10 15:56:29.693768 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:29.693735 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:29.693947 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:29.693771 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:29.693947 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:29.693855 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:29.693947 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:29.693870 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:29.694110 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:29.693945 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:29.694110 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:29.694052 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:30.239264 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:30.239229 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sms7n_7438a602-18a4-48c3-aa99-7a3cdd98cae7/node-ca/0.log" Sep 10 15:56:31.330308 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.330285 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:31.330608 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.330394 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:31.330608 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.330437 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs podName:43c09421-c94c-4ec6-abe4-a30be09975c6 nodeName:}" failed. No retries permitted until 2026-09-10 15:57:03.330425498 +0000 UTC m=+66.106483913 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs") pod "network-metrics-daemon-jclb4" (UID: "43c09421-c94c-4ec6-abe4-a30be09975c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 15:56:31.430769 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.430733 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:31.430909 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.430876 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:31.430948 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.430935 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret podName:466e913c-b578-40aa-b099-669bdfe2ecd5 nodeName:}" failed. No retries permitted until 2026-09-10 15:57:03.430918373 +0000 UTC m=+66.206976804 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret") pod "global-pull-secret-syncer-q49dm" (UID: "466e913c-b578-40aa-b099-669bdfe2ecd5") : object "kube-system"/"original-pull-secret" not registered Sep 10 15:56:31.531792 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.531731 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:31.531894 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.531860 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 15:56:31.531894 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.531881 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 15:56:31.531894 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.531892 2576 projected.go:196] Error preparing data for projected volume kube-api-access-5qh7f for pod openshift-network-diagnostics/network-check-target-mwvrc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:31.532040 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.531964 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f podName:67089a15-730b-4bc3-84ad-a429e8edc246 nodeName:}" failed. No retries permitted until 2026-09-10 15:57:03.531934144 +0000 UTC m=+66.307992557 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qh7f" (UniqueName: "kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f") pod "network-check-target-mwvrc" (UID: "67089a15-730b-4bc3-84ad-a429e8edc246") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 15:56:31.693887 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.693859 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:31.693987 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.693859 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:31.694081 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.693970 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jclb4" podUID="43c09421-c94c-4ec6-abe4-a30be09975c6" Sep 10 15:56:31.694135 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.694076 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q49dm" podUID="466e913c-b578-40aa-b099-669bdfe2ecd5" Sep 10 15:56:31.694135 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.694120 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:31.694231 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:56:31.694213 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mwvrc" podUID="67089a15-730b-4bc3-84ad-a429e8edc246" Sep 10 15:56:31.924479 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.924411 2576 generic.go:356] "Generic (PLEG): container finished" podID="e92b2fbf-b2f0-476b-842a-93d60ce94b8c" containerID="493119090d2e295befbbda06a73db73cbca58e577154f18c2fdf8f6598ae0aad" exitCode=0 Sep 10 15:56:31.924479 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.924469 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" event={"ID":"e92b2fbf-b2f0-476b-842a-93d60ce94b8c","Type":"ContainerDied","Data":"493119090d2e295befbbda06a73db73cbca58e577154f18c2fdf8f6598ae0aad"} Sep 10 15:56:31.990830 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.990807 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-74.ec2.internal" event="NodeReady" Sep 10 15:56:31.990948 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:31.990910 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 15:56:32.929628 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:32.929594 2576 generic.go:356] "Generic (PLEG): container finished" podID="e92b2fbf-b2f0-476b-842a-93d60ce94b8c" containerID="c04517b8c9f3c953d63f6910cf435993c185a54f86f8a565976031d14787190a" exitCode=0 Sep 10 15:56:32.930164 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:32.929643 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" event={"ID":"e92b2fbf-b2f0-476b-842a-93d60ce94b8c","Type":"ContainerDied","Data":"c04517b8c9f3c953d63f6910cf435993c185a54f86f8a565976031d14787190a"} Sep 10 15:56:33.694358 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.694327 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:56:33.694493 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.694327 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:56:33.694570 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.694334 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:56:33.700494 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.700473 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 15:56:33.700701 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.700684 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 15:56:33.700829 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.700813 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 15:56:33.702023 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.701999 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5znt\"" Sep 10 15:56:33.702124 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.702043 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 15:56:33.702124 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.702056 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v658w\"" Sep 10 15:56:33.935258 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.935223 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" event={"ID":"e92b2fbf-b2f0-476b-842a-93d60ce94b8c","Type":"ContainerStarted","Data":"20819a6306314dbc8a97248f7108fa08b0f7e083e745525f5e13242013e710e3"} Sep 10 15:56:33.971046 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:33.970964 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nq9fq" podStartSLOduration=7.914613903 podStartE2EDuration="35.970951838s" totalImagesPullingTime="28.056337935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:33.9707906 +0000 UTC m=+36.746849053" watchObservedRunningTime="2026-09-10 15:56:33.970951838 +0000 UTC m=+36.747010269" Sep 10 15:56:56.920358 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:56:56.920324 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zwpx" Sep 10 15:57:02.039561 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.039523 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4m2cz"] Sep 10 15:57:02.074852 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.074824 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4m2cz" Sep 10 15:57:02.078626 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.078604 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 15:57:02.078741 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.078653 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 15:57:02.079414 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.079390 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 15:57:02.079491 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.079397 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xwnwk\"" Sep 10 15:57:02.080780 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.080760 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4m2cz"] Sep 10 15:57:02.141849 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.141823 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87nkp\" (UniqueName: \"kubernetes.io/projected/b703f5eb-7202-4225-8193-a0b36bc187d6-kube-api-access-87nkp\") pod \"ingress-canary-4m2cz\" (UID: \"b703f5eb-7202-4225-8193-a0b36bc187d6\") " pod="openshift-ingress-canary/ingress-canary-4m2cz" Sep 10 15:57:02.141956 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.141873 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b703f5eb-7202-4225-8193-a0b36bc187d6-cert\") pod \"ingress-canary-4m2cz\" (UID: \"b703f5eb-7202-4225-8193-a0b36bc187d6\") " pod="openshift-ingress-canary/ingress-canary-4m2cz" Sep 10 15:57:02.181151 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.181129 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj"] Sep 10 15:57:02.201738 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.201697 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tjh7r"] Sep 10 15:57:02.201856 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.201838 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" Sep 10 15:57:02.204871 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.204852 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qbr5j\"" Sep 10 15:57:02.204950 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.204852 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 15:57:02.217607 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.217582 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x"] Sep 10 15:57:02.217700 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.217644 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.223588 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.223542 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 15:57:02.223765 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.223597 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 15:57:02.223765 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.223610 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fvhfz\"" Sep 10 15:57:02.231649 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.231631 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv"] Sep 10 15:57:02.231785 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.231770 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.242515 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.242494 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b703f5eb-7202-4225-8193-a0b36bc187d6-cert\") pod \"ingress-canary-4m2cz\" (UID: \"b703f5eb-7202-4225-8193-a0b36bc187d6\") " pod="openshift-ingress-canary/ingress-canary-4m2cz" Sep 10 15:57:02.242609 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.242523 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f11d3e7f-f422-46de-aff9-fcd9448b183f-metrics-tls\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.242609 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.242547 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rdf2\" (UniqueName: \"kubernetes.io/projected/f11d3e7f-f422-46de-aff9-fcd9448b183f-kube-api-access-2rdf2\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.242609 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.242564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f11d3e7f-f422-46de-aff9-fcd9448b183f-config-volume\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.242781 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.242680 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a6bde5d3-367d-449b-b159-e32ac7611052-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2wjvj\" (UID: \"a6bde5d3-367d-449b-b159-e32ac7611052\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" Sep 10 15:57:02.242781 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.242736 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f11d3e7f-f422-46de-aff9-fcd9448b183f-tmp-dir\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.242849 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.242793 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87nkp\" (UniqueName: \"kubernetes.io/projected/b703f5eb-7202-4225-8193-a0b36bc187d6-kube-api-access-87nkp\") pod \"ingress-canary-4m2cz\" (UID: \"b703f5eb-7202-4225-8193-a0b36bc187d6\") " pod="openshift-ingress-canary/ingress-canary-4m2cz" Sep 10 15:57:02.246478 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.246463 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b703f5eb-7202-4225-8193-a0b36bc187d6-cert\") pod \"ingress-canary-4m2cz\" (UID: \"b703f5eb-7202-4225-8193-a0b36bc187d6\") " pod="openshift-ingress-canary/ingress-canary-4m2cz" Sep 10 15:57:02.247718 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.247693 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-59zmf"] Sep 10 15:57:02.247789 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.247776 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.267981 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.267964 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv"] Sep 10 15:57:02.268062 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.267989 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tjh7r"] Sep 10 15:57:02.268062 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.268012 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj"] Sep 10 15:57:02.268062 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.268031 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x"] Sep 10 15:57:02.268162 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.268069 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.276918 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.276901 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 10 15:57:02.288262 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.288239 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 10 15:57:02.288628 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.288575 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 10 15:57:02.288722 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.288614 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 10 15:57:02.288829 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.288814 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 10 15:57:02.288876 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.288860 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 10 15:57:02.288927 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.288876 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 10 15:57:02.289129 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.289116 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 15:57:02.290207 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.290162 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 15:57:02.290811 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.290794 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-59zmf"] Sep 10 15:57:02.292112 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.292099 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 15:57:02.292167 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.292103 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zcsvb\"" Sep 10 15:57:02.292167 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.292105 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 10 15:57:02.293418 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.293401 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 15:57:02.343984 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.343963 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-ca\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.344086 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.343992 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a6bde5d3-367d-449b-b159-e32ac7611052-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2wjvj\" (UID: \"a6bde5d3-367d-449b-b159-e32ac7611052\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" Sep 10 15:57:02.344086 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344009 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f11d3e7f-f422-46de-aff9-fcd9448b183f-tmp-dir\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.344086 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344029 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d96b9\" (UniqueName: \"kubernetes.io/projected/0483393d-32f7-42f7-96c4-1f52ab594fcd-kube-api-access-d96b9\") pod \"klusterlet-addon-workmgr-55bd4ddd74-7zpqv\" (UID: \"0483393d-32f7-42f7-96c4-1f52ab594fcd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.344086 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344047 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-hub\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.344281 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/35285e79-c766-484e-982b-85f498f2d02c-data-volume\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.344281 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344146 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.344281 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344176 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0483393d-32f7-42f7-96c4-1f52ab594fcd-klusterlet-config\") pod \"klusterlet-addon-workmgr-55bd4ddd74-7zpqv\" (UID: \"0483393d-32f7-42f7-96c4-1f52ab594fcd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.344281 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344194 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f11d3e7f-f422-46de-aff9-fcd9448b183f-metrics-tls\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.344281 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344232 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/35285e79-c766-484e-982b-85f498f2d02c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.344497 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344281 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de66926-bb30-43cb-8230-d147f2cd69f6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.344497 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344310 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f11d3e7f-f422-46de-aff9-fcd9448b183f-tmp-dir\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.344497 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344325 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/35285e79-c766-484e-982b-85f498f2d02c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.344497 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glktd\" (UniqueName: \"kubernetes.io/projected/35285e79-c766-484e-982b-85f498f2d02c-kube-api-access-glktd\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.344497 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344389 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rdf2\" (UniqueName: \"kubernetes.io/projected/f11d3e7f-f422-46de-aff9-fcd9448b183f-kube-api-access-2rdf2\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.344497 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344423 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f11d3e7f-f422-46de-aff9-fcd9448b183f-config-volume\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.344497 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344484 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.344804 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344514 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2zxq\" (UniqueName: \"kubernetes.io/projected/0de66926-bb30-43cb-8230-d147f2cd69f6-kube-api-access-g2zxq\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.344804 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344540 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0483393d-32f7-42f7-96c4-1f52ab594fcd-tmp\") pod \"klusterlet-addon-workmgr-55bd4ddd74-7zpqv\" (UID: \"0483393d-32f7-42f7-96c4-1f52ab594fcd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.344804 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344584 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/35285e79-c766-484e-982b-85f498f2d02c-crio-socket\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.344986 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.344967 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f11d3e7f-f422-46de-aff9-fcd9448b183f-config-volume\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.346957 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.346933 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f11d3e7f-f422-46de-aff9-fcd9448b183f-metrics-tls\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.347060 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.346958 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a6bde5d3-367d-449b-b159-e32ac7611052-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2wjvj\" (UID: \"a6bde5d3-367d-449b-b159-e32ac7611052\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" Sep 10 15:57:02.377297 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.377268 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87nkp\" (UniqueName: \"kubernetes.io/projected/b703f5eb-7202-4225-8193-a0b36bc187d6-kube-api-access-87nkp\") pod \"ingress-canary-4m2cz\" (UID: \"b703f5eb-7202-4225-8193-a0b36bc187d6\") " pod="openshift-ingress-canary/ingress-canary-4m2cz" Sep 10 15:57:02.384068 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.384049 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4m2cz" Sep 10 15:57:02.387540 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.387517 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rdf2\" (UniqueName: \"kubernetes.io/projected/f11d3e7f-f422-46de-aff9-fcd9448b183f-kube-api-access-2rdf2\") pod \"dns-default-tjh7r\" (UID: \"f11d3e7f-f422-46de-aff9-fcd9448b183f\") " pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.445763 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445732 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0483393d-32f7-42f7-96c4-1f52ab594fcd-tmp\") pod \"klusterlet-addon-workmgr-55bd4ddd74-7zpqv\" (UID: \"0483393d-32f7-42f7-96c4-1f52ab594fcd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.445906 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445776 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/35285e79-c766-484e-982b-85f498f2d02c-crio-socket\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.445906 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445806 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-ca\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.445906 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445842 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d96b9\" (UniqueName: \"kubernetes.io/projected/0483393d-32f7-42f7-96c4-1f52ab594fcd-kube-api-access-d96b9\") pod \"klusterlet-addon-workmgr-55bd4ddd74-7zpqv\" (UID: \"0483393d-32f7-42f7-96c4-1f52ab594fcd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.445906 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-hub\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.446101 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445915 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/35285e79-c766-484e-982b-85f498f2d02c-data-volume\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.446101 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445942 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.446101 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445971 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0483393d-32f7-42f7-96c4-1f52ab594fcd-klusterlet-config\") pod \"klusterlet-addon-workmgr-55bd4ddd74-7zpqv\" (UID: \"0483393d-32f7-42f7-96c4-1f52ab594fcd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.446101 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445996 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/35285e79-c766-484e-982b-85f498f2d02c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.446101 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.446021 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de66926-bb30-43cb-8230-d147f2cd69f6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.446101 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.446047 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/35285e79-c766-484e-982b-85f498f2d02c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.446101 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.446076 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glktd\" (UniqueName: \"kubernetes.io/projected/35285e79-c766-484e-982b-85f498f2d02c-kube-api-access-glktd\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.446429 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.445910 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/35285e79-c766-484e-982b-85f498f2d02c-crio-socket\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.446429 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.446155 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.446429 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.446185 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2zxq\" (UniqueName: \"kubernetes.io/projected/0de66926-bb30-43cb-8230-d147f2cd69f6-kube-api-access-g2zxq\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.446429 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.446232 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/35285e79-c766-484e-982b-85f498f2d02c-data-volume\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.446640 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.446075 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0483393d-32f7-42f7-96c4-1f52ab594fcd-tmp\") pod \"klusterlet-addon-workmgr-55bd4ddd74-7zpqv\" (UID: \"0483393d-32f7-42f7-96c4-1f52ab594fcd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.447159 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.447086 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de66926-bb30-43cb-8230-d147f2cd69f6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.447159 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.447138 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/35285e79-c766-484e-982b-85f498f2d02c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.448555 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.448502 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-hub\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.448662 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.448632 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-ca\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.449229 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.449010 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0483393d-32f7-42f7-96c4-1f52ab594fcd-klusterlet-config\") pod \"klusterlet-addon-workmgr-55bd4ddd74-7zpqv\" (UID: \"0483393d-32f7-42f7-96c4-1f52ab594fcd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.449229 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.449124 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/35285e79-c766-484e-982b-85f498f2d02c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.449229 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.449191 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.449756 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.449739 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0de66926-bb30-43cb-8230-d147f2cd69f6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.477095 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.477070 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glktd\" (UniqueName: \"kubernetes.io/projected/35285e79-c766-484e-982b-85f498f2d02c-kube-api-access-glktd\") pod \"insights-runtime-extractor-59zmf\" (UID: \"35285e79-c766-484e-982b-85f498f2d02c\") " pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.477769 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.477750 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d96b9\" (UniqueName: \"kubernetes.io/projected/0483393d-32f7-42f7-96c4-1f52ab594fcd-kube-api-access-d96b9\") pod \"klusterlet-addon-workmgr-55bd4ddd74-7zpqv\" (UID: \"0483393d-32f7-42f7-96c4-1f52ab594fcd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.478526 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.478500 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2zxq\" (UniqueName: \"kubernetes.io/projected/0de66926-bb30-43cb-8230-d147f2cd69f6-kube-api-access-g2zxq\") pod \"cluster-proxy-proxy-agent-656865976-rxb5x\" (UID: \"0de66926-bb30-43cb-8230-d147f2cd69f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.509950 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.509926 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" Sep 10 15:57:02.527631 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.527613 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:02.554352 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.554163 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" Sep 10 15:57:02.564864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.564834 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:02.576186 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.575846 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-59zmf" Sep 10 15:57:02.695625 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.695598 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4m2cz"] Sep 10 15:57:02.698967 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:02.698944 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb703f5eb_7202_4225_8193_a0b36bc187d6.slice/crio-43ddaae4e4a896c63af5625ba05dfa1912d0bdee55ea5920318bd3a405c91d15 WatchSource:0}: Error finding container 43ddaae4e4a896c63af5625ba05dfa1912d0bdee55ea5920318bd3a405c91d15: Status 404 returned error can't find the container with id 43ddaae4e4a896c63af5625ba05dfa1912d0bdee55ea5920318bd3a405c91d15 Sep 10 15:57:02.811324 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.811299 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj"] Sep 10 15:57:02.812214 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:02.812196 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6bde5d3_367d_449b_b159_e32ac7611052.slice/crio-66dbf11c7ecd3222d3b4c2ca7a323bafbf4916056604cfcf5261509d40bf34c3 WatchSource:0}: Error finding container 66dbf11c7ecd3222d3b4c2ca7a323bafbf4916056604cfcf5261509d40bf34c3: Status 404 returned error can't find the container with id 66dbf11c7ecd3222d3b4c2ca7a323bafbf4916056604cfcf5261509d40bf34c3 Sep 10 15:57:02.864554 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:02.864507 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf11d3e7f_f422_46de_aff9_fcd9448b183f.slice/crio-c681eac161750998d68e4901ce4fdf50b654951c75cbdf83712263d14bb12973 WatchSource:0}: Error finding container c681eac161750998d68e4901ce4fdf50b654951c75cbdf83712263d14bb12973: Status 404 returned error can't find the container with id c681eac161750998d68e4901ce4fdf50b654951c75cbdf83712263d14bb12973 Sep 10 15:57:02.866383 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.866360 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tjh7r"] Sep 10 15:57:02.867419 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.867335 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x"] Sep 10 15:57:02.867692 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:02.867671 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0de66926_bb30_43cb_8230_d147f2cd69f6.slice/crio-68185c852f33c0ffe6dcf6bca7879eaae80f5ee08bac3e9abc3605c655f6ecd1 WatchSource:0}: Error finding container 68185c852f33c0ffe6dcf6bca7879eaae80f5ee08bac3e9abc3605c655f6ecd1: Status 404 returned error can't find the container with id 68185c852f33c0ffe6dcf6bca7879eaae80f5ee08bac3e9abc3605c655f6ecd1 Sep 10 15:57:02.902933 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.902912 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv"] Sep 10 15:57:02.906185 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:02.906160 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0483393d_32f7_42f7_96c4_1f52ab594fcd.slice/crio-a3c772700325f390dfbf8f4c72063856925e53e110c6d73725a96d934c02665a WatchSource:0}: Error finding container a3c772700325f390dfbf8f4c72063856925e53e110c6d73725a96d934c02665a: Status 404 returned error can't find the container with id a3c772700325f390dfbf8f4c72063856925e53e110c6d73725a96d934c02665a Sep 10 15:57:02.906820 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:02.906799 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-59zmf"] Sep 10 15:57:02.909334 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:02.909313 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35285e79_c766_484e_982b_85f498f2d02c.slice/crio-3f7c31144cd8a8e77f34c8e1ea5d522f0067c2b7d3ae0eb2ecfac661571f8850 WatchSource:0}: Error finding container 3f7c31144cd8a8e77f34c8e1ea5d522f0067c2b7d3ae0eb2ecfac661571f8850: Status 404 returned error can't find the container with id 3f7c31144cd8a8e77f34c8e1ea5d522f0067c2b7d3ae0eb2ecfac661571f8850 Sep 10 15:57:03.004553 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.004523 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" event={"ID":"0483393d-32f7-42f7-96c4-1f52ab594fcd","Type":"ContainerStarted","Data":"a3c772700325f390dfbf8f4c72063856925e53e110c6d73725a96d934c02665a"} Sep 10 15:57:03.005536 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.005514 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-59zmf" event={"ID":"35285e79-c766-484e-982b-85f498f2d02c","Type":"ContainerStarted","Data":"3f7c31144cd8a8e77f34c8e1ea5d522f0067c2b7d3ae0eb2ecfac661571f8850"} Sep 10 15:57:03.006534 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.006516 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" event={"ID":"a6bde5d3-367d-449b-b159-e32ac7611052","Type":"ContainerStarted","Data":"66dbf11c7ecd3222d3b4c2ca7a323bafbf4916056604cfcf5261509d40bf34c3"} Sep 10 15:57:03.007583 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.007566 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4m2cz" event={"ID":"b703f5eb-7202-4225-8193-a0b36bc187d6","Type":"ContainerStarted","Data":"43ddaae4e4a896c63af5625ba05dfa1912d0bdee55ea5920318bd3a405c91d15"} Sep 10 15:57:03.008592 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.008572 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" event={"ID":"0de66926-bb30-43cb-8230-d147f2cd69f6","Type":"ContainerStarted","Data":"68185c852f33c0ffe6dcf6bca7879eaae80f5ee08bac3e9abc3605c655f6ecd1"} Sep 10 15:57:03.009539 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.009518 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tjh7r" event={"ID":"f11d3e7f-f422-46de-aff9-fcd9448b183f","Type":"ContainerStarted","Data":"c681eac161750998d68e4901ce4fdf50b654951c75cbdf83712263d14bb12973"} Sep 10 15:57:03.353138 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.353102 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:57:03.358574 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.358221 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 15:57:03.368548 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.367564 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c09421-c94c-4ec6-abe4-a30be09975c6-metrics-certs\") pod \"network-metrics-daemon-jclb4\" (UID: \"43c09421-c94c-4ec6-abe4-a30be09975c6\") " pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:57:03.407760 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.407730 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5znt\"" Sep 10 15:57:03.414492 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.414436 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jclb4" Sep 10 15:57:03.454317 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.454280 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:57:03.457338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.457312 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 15:57:03.468227 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.468149 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/466e913c-b578-40aa-b099-669bdfe2ecd5-original-pull-secret\") pod \"global-pull-secret-syncer-q49dm\" (UID: \"466e913c-b578-40aa-b099-669bdfe2ecd5\") " pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:57:03.554952 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.554915 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:57:03.562831 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.562805 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 15:57:03.568430 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.568379 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 15:57:03.586269 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.586243 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qh7f\" (UniqueName: \"kubernetes.io/projected/67089a15-730b-4bc3-84ad-a429e8edc246-kube-api-access-5qh7f\") pod \"network-check-target-mwvrc\" (UID: \"67089a15-730b-4bc3-84ad-a429e8edc246\") " pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:57:03.712557 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.712522 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v658w\"" Sep 10 15:57:03.713920 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.713898 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q49dm" Sep 10 15:57:03.720414 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.720394 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:57:03.723173 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:03.723149 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jclb4"] Sep 10 15:57:03.841394 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:03.841343 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c09421_c94c_4ec6_abe4_a30be09975c6.slice/crio-819ebfa9c3bb4ec96d9e0c6f6bb9edc693655a7f142c1e1c5a702ce490681e57 WatchSource:0}: Error finding container 819ebfa9c3bb4ec96d9e0c6f6bb9edc693655a7f142c1e1c5a702ce490681e57: Status 404 returned error can't find the container with id 819ebfa9c3bb4ec96d9e0c6f6bb9edc693655a7f142c1e1c5a702ce490681e57 Sep 10 15:57:04.018900 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:04.018751 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-59zmf" event={"ID":"35285e79-c766-484e-982b-85f498f2d02c","Type":"ContainerStarted","Data":"d4bd7059d111ae90112cea75fdcef5649b8121446f475a4e9aaeaf4ddd1f95b0"} Sep 10 15:57:04.021396 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:04.021209 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jclb4" event={"ID":"43c09421-c94c-4ec6-abe4-a30be09975c6","Type":"ContainerStarted","Data":"819ebfa9c3bb4ec96d9e0c6f6bb9edc693655a7f142c1e1c5a702ce490681e57"} Sep 10 15:57:04.154013 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:04.153986 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mwvrc"] Sep 10 15:57:04.156309 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:04.156232 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67089a15_730b_4bc3_84ad_a429e8edc246.slice/crio-de109d4cde0acb68ff539ed5234ab065a3913bd0a36ad0dbbf97c14eaaef131b WatchSource:0}: Error finding container de109d4cde0acb68ff539ed5234ab065a3913bd0a36ad0dbbf97c14eaaef131b: Status 404 returned error can't find the container with id de109d4cde0acb68ff539ed5234ab065a3913bd0a36ad0dbbf97c14eaaef131b Sep 10 15:57:04.178554 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:04.178530 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q49dm"] Sep 10 15:57:05.039042 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:05.038942 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mwvrc" event={"ID":"67089a15-730b-4bc3-84ad-a429e8edc246","Type":"ContainerStarted","Data":"de109d4cde0acb68ff539ed5234ab065a3913bd0a36ad0dbbf97c14eaaef131b"} Sep 10 15:57:05.051272 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:05.051172 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-59zmf" event={"ID":"35285e79-c766-484e-982b-85f498f2d02c","Type":"ContainerStarted","Data":"da33804f50df0f18a80b0e1b9e40245a5e89ab4ee306c8bd123bd625cfdacff6"} Sep 10 15:57:05.056902 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:05.056834 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q49dm" event={"ID":"466e913c-b578-40aa-b099-669bdfe2ecd5","Type":"ContainerStarted","Data":"98f337b0be23ac9da6500b6690965c972494a5f9fcd09916987da5c8e2114310"} Sep 10 15:57:06.232461 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.231368 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f4d64bc46-ghgcn"] Sep 10 15:57:06.235607 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.233687 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.239406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.238892 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 15:57:06.241301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.240393 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 15:57:06.241301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.240536 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 15:57:06.241301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.240645 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 15:57:06.241301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.240781 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 15:57:06.241301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.240910 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jkblw\"" Sep 10 15:57:06.241301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.240986 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 15:57:06.241301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.241089 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 15:57:06.255789 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.255404 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f4d64bc46-ghgcn"] Sep 10 15:57:06.277251 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.276929 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-oauth-config\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.277251 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.276975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpgdn\" (UniqueName: \"kubernetes.io/projected/4e8f8b2b-7866-4de8-986b-be57780d888e-kube-api-access-bpgdn\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.277251 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.277036 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-serving-cert\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.277251 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.277061 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-console-config\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.277251 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.277083 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-oauth-serving-cert\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.277251 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.277134 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-service-ca\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.379074 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.378139 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-serving-cert\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.379074 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.378184 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-console-config\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.379074 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.378213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-oauth-serving-cert\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.379074 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.378306 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-service-ca\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.379074 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.378342 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-oauth-config\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.379074 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.378376 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpgdn\" (UniqueName: \"kubernetes.io/projected/4e8f8b2b-7866-4de8-986b-be57780d888e-kube-api-access-bpgdn\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.379512 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.379485 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-oauth-serving-cert\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.380341 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.380056 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-service-ca\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.380534 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.380511 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-console-config\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.382940 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.382822 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-oauth-config\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.385622 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.385600 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-serving-cert\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.392871 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.392831 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpgdn\" (UniqueName: \"kubernetes.io/projected/4e8f8b2b-7866-4de8-986b-be57780d888e-kube-api-access-bpgdn\") pod \"console-6f4d64bc46-ghgcn\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:06.548024 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:06.547631 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:14.343139 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:14.343036 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f4d64bc46-ghgcn"] Sep 10 15:57:14.351198 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:14.351162 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8f8b2b_7866_4de8_986b_be57780d888e.slice/crio-9bd2904b51a8b6c4a4ae8e8f02fbc7bf9218946fb8f18cb76595719b3e2b67a0 WatchSource:0}: Error finding container 9bd2904b51a8b6c4a4ae8e8f02fbc7bf9218946fb8f18cb76595719b3e2b67a0: Status 404 returned error can't find the container with id 9bd2904b51a8b6c4a4ae8e8f02fbc7bf9218946fb8f18cb76595719b3e2b67a0 Sep 10 15:57:15.094199 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.094161 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tjh7r" event={"ID":"f11d3e7f-f422-46de-aff9-fcd9448b183f","Type":"ContainerStarted","Data":"6295bd03d22588a2c6bdd7bd00c4bdf052764ebb5f181b4f4c945e2d86446f5c"} Sep 10 15:57:15.094382 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.094221 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tjh7r" event={"ID":"f11d3e7f-f422-46de-aff9-fcd9448b183f","Type":"ContainerStarted","Data":"4b0aa32a377dac085936c62ad7cae1236c3f6e3c3f7ba7ac71594f4bbaada8d1"} Sep 10 15:57:15.094382 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.094321 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:15.095774 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.095744 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mwvrc" event={"ID":"67089a15-730b-4bc3-84ad-a429e8edc246","Type":"ContainerStarted","Data":"20facf71ed15981627a3a444d13a61c6bb2147483da577458b245ef49cffcce8"} Sep 10 15:57:15.095907 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.095834 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:57:15.097237 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.097215 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" event={"ID":"0483393d-32f7-42f7-96c4-1f52ab594fcd","Type":"ContainerStarted","Data":"0232ef6d5ade28e0a5c3b8db3df9288f7a5f8b8bdd13457f64e4c9ee5cc329f5"} Sep 10 15:57:15.097365 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.097349 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:15.099133 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.099114 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" Sep 10 15:57:15.099487 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.099469 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-59zmf" event={"ID":"35285e79-c766-484e-982b-85f498f2d02c","Type":"ContainerStarted","Data":"9a4209fbe8dd53cd9d3cc8e9d0dd120e8f8dc6d185ca4d79ef41d456dbceee45"} Sep 10 15:57:15.100925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.100907 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" event={"ID":"a6bde5d3-367d-449b-b159-e32ac7611052","Type":"ContainerStarted","Data":"14b2647572fe7e97c52a5ac7b7e4e1b89d3d8d32ab4600e4d969ec711c7a82c0"} Sep 10 15:57:15.101111 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.101092 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" Sep 10 15:57:15.102419 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.102395 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4m2cz" event={"ID":"b703f5eb-7202-4225-8193-a0b36bc187d6","Type":"ContainerStarted","Data":"a02bed05edeb6291692ecb7faa1842a49444e808b270e72f43bb48262bbdbfad"} Sep 10 15:57:15.104271 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.104248 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jclb4" event={"ID":"43c09421-c94c-4ec6-abe4-a30be09975c6","Type":"ContainerStarted","Data":"527182826b74390283861f06294eff56d2bce2a11e982b083aa6d1335ff987b8"} Sep 10 15:57:15.104370 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.104288 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jclb4" event={"ID":"43c09421-c94c-4ec6-abe4-a30be09975c6","Type":"ContainerStarted","Data":"604cc0abe41203ca221300ba39d25efeaef8aaaac293f6a4f943bf730934f971"} Sep 10 15:57:15.105861 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.105730 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q49dm" event={"ID":"466e913c-b578-40aa-b099-669bdfe2ecd5","Type":"ContainerStarted","Data":"429b58c09b65a21b9d9d02a01cc35130080442e0483a3906f22464ea2ad4232f"} Sep 10 15:57:15.106246 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.106206 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" Sep 10 15:57:15.107086 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.107066 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4d64bc46-ghgcn" event={"ID":"4e8f8b2b-7866-4de8-986b-be57780d888e","Type":"ContainerStarted","Data":"9bd2904b51a8b6c4a4ae8e8f02fbc7bf9218946fb8f18cb76595719b3e2b67a0"} Sep 10 15:57:15.108981 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.108962 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" event={"ID":"0de66926-bb30-43cb-8230-d147f2cd69f6","Type":"ContainerStarted","Data":"7514d53496bcde00706e2f814461ca9617f201de218156ba7c54824b6969c719"} Sep 10 15:57:15.109049 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.108993 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" event={"ID":"0de66926-bb30-43cb-8230-d147f2cd69f6","Type":"ContainerStarted","Data":"4884f7cf3812d0d85568f242858a87f2e6311505ebc84c6bd855d6c93b920a39"} Sep 10 15:57:15.109049 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.109012 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" event={"ID":"0de66926-bb30-43cb-8230-d147f2cd69f6","Type":"ContainerStarted","Data":"9eb86d5739ab09026834956a246ec11da9de0cc0208da7ceee15565687a1cf37"} Sep 10 15:57:15.126192 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.126022 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tjh7r" podStartSLOduration=3.740343257 podStartE2EDuration="13.126006914s" totalImagesPullingTime="9.385663657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:15.125813832 +0000 UTC m=+77.901872278" watchObservedRunningTime="2026-09-10 15:57:15.126006914 +0000 UTC m=+77.902065344" Sep 10 15:57:15.165338 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.165294 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-59zmf" podStartSLOduration=2.268392882 podStartE2EDuration="13.165283641s" totalImagesPullingTime="10.896890759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:57:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:15.165169404 +0000 UTC m=+77.941227837" watchObservedRunningTime="2026-09-10 15:57:15.165283641 +0000 UTC m=+77.941342073" Sep 10 15:57:15.191022 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.190986 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mwvrc" podStartSLOduration=67.298222436 podStartE2EDuration="1m17.190970418s" totalImagesPullingTime="9.892747982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:15.19010136 +0000 UTC m=+77.966159791" watchObservedRunningTime="2026-09-10 15:57:15.190970418 +0000 UTC m=+77.967028850" Sep 10 15:57:15.217147 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.214546 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q49dm" podStartSLOduration=67.334202524 podStartE2EDuration="1m17.214531002s" totalImagesPullingTime="9.880328478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:15.213689178 +0000 UTC m=+77.989747608" watchObservedRunningTime="2026-09-10 15:57:15.214531002 +0000 UTC m=+77.990589435" Sep 10 15:57:15.287404 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.287339 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4m2cz" podStartSLOduration=1.991633867 podStartE2EDuration="13.287317935s" totalImagesPullingTime="11.295684068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:15.250781953 +0000 UTC m=+78.026840396" watchObservedRunningTime="2026-09-10 15:57:15.287317935 +0000 UTC m=+78.063376359" Sep 10 15:57:15.287603 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.287451 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656865976-rxb5x" podStartSLOduration=2.105279046 podStartE2EDuration="13.287438862s" totalImagesPullingTime="11.182159816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:15.286356956 +0000 UTC m=+78.062415401" watchObservedRunningTime="2026-09-10 15:57:15.287438862 +0000 UTC m=+78.063497295" Sep 10 15:57:15.336608 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.336560 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2wjvj" podStartSLOduration=3.952920706 podStartE2EDuration="13.336549754s" totalImagesPullingTime="9.383629048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:15.307321983 +0000 UTC m=+78.083380416" watchObservedRunningTime="2026-09-10 15:57:15.336549754 +0000 UTC m=+78.112608185" Sep 10 15:57:15.336758 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.336725 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55bd4ddd74-7zpqv" podStartSLOduration=2.184845824 podStartE2EDuration="13.336704115s" totalImagesPullingTime="11.151858291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:15.335366323 +0000 UTC m=+78.111424759" watchObservedRunningTime="2026-09-10 15:57:15.336704115 +0000 UTC m=+78.112762546" Sep 10 15:57:15.360281 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:15.360203 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jclb4" podStartSLOduration=67.162525388 podStartE2EDuration="1m17.360188445s" totalImagesPullingTime="10.197663057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:15.35897682 +0000 UTC m=+78.135035257" watchObservedRunningTime="2026-09-10 15:57:15.360188445 +0000 UTC m=+78.136246877" Sep 10 15:57:18.121976 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:18.121934 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4d64bc46-ghgcn" event={"ID":"4e8f8b2b-7866-4de8-986b-be57780d888e","Type":"ContainerStarted","Data":"44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373"} Sep 10 15:57:25.117497 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.117369 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tjh7r" Sep 10 15:57:25.146250 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.146198 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f4d64bc46-ghgcn" podStartSLOduration=16.155636078 podStartE2EDuration="19.146183858s" totalImagesPullingTime="2.99054778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:18.141841688 +0000 UTC m=+80.917900119" watchObservedRunningTime="2026-09-10 15:57:25.146183858 +0000 UTC m=+87.922242290" Sep 10 15:57:25.557277 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.557236 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b88jf"] Sep 10 15:57:25.561575 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.561559 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.565048 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.565024 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 15:57:25.565344 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.565331 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 15:57:25.566303 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.566289 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 15:57:25.566349 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.566298 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 15:57:25.566542 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.566521 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dd9s6\"" Sep 10 15:57:25.566639 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.566563 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 15:57:25.566639 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.566597 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 15:57:25.616139 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.616115 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1fc51db-e496-4b33-989b-e2b1e8c10890-root\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.616243 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.616142 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-textfile\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.616243 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.616171 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-wtmp\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.616243 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.616215 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1fc51db-e496-4b33-989b-e2b1e8c10890-sys\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.616392 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.616279 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1fc51db-e496-4b33-989b-e2b1e8c10890-metrics-client-ca\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.616392 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.616307 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-accelerators-collector-config\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.616392 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.616334 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtvtj\" (UniqueName: \"kubernetes.io/projected/e1fc51db-e496-4b33-989b-e2b1e8c10890-kube-api-access-dtvtj\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.616557 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.616384 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.616557 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.616423 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-tls\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717055 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717032 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1fc51db-e496-4b33-989b-e2b1e8c10890-metrics-client-ca\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717133 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717057 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-accelerators-collector-config\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717133 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717073 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtvtj\" (UniqueName: \"kubernetes.io/projected/e1fc51db-e496-4b33-989b-e2b1e8c10890-kube-api-access-dtvtj\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717133 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717089 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717133 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717105 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-tls\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717285 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717144 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1fc51db-e496-4b33-989b-e2b1e8c10890-root\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717285 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717171 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-textfile\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717285 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717206 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-wtmp\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717285 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:57:25.717227 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 10 15:57:25.717285 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717272 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1fc51db-e496-4b33-989b-e2b1e8c10890-sys\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717468 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:57:25.717295 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-tls podName:e1fc51db-e496-4b33-989b-e2b1e8c10890 nodeName:}" failed. No retries permitted until 2026-09-10 15:57:26.217275893 +0000 UTC m=+88.993334320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-tls") pod "node-exporter-b88jf" (UID: "e1fc51db-e496-4b33-989b-e2b1e8c10890") : secret "node-exporter-tls" not found Sep 10 15:57:25.717468 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717230 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1fc51db-e496-4b33-989b-e2b1e8c10890-sys\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717468 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717318 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1fc51db-e496-4b33-989b-e2b1e8c10890-root\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717621 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717478 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-wtmp\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717670 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717632 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-textfile\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717670 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717642 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-accelerators-collector-config\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.717794 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.717675 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1fc51db-e496-4b33-989b-e2b1e8c10890-metrics-client-ca\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.719414 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.719392 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:25.730301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:25.730270 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtvtj\" (UniqueName: \"kubernetes.io/projected/e1fc51db-e496-4b33-989b-e2b1e8c10890-kube-api-access-dtvtj\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:26.220793 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:26.220758 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-tls\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:26.223058 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:26.223032 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1fc51db-e496-4b33-989b-e2b1e8c10890-node-exporter-tls\") pod \"node-exporter-b88jf\" (UID: \"e1fc51db-e496-4b33-989b-e2b1e8c10890\") " pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:26.470190 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:26.470155 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b88jf" Sep 10 15:57:26.480652 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:26.480628 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1fc51db_e496_4b33_989b_e2b1e8c10890.slice/crio-2e919e4c3e4359384731296d80612db41ca3e9d44b448f3fade1eabf01df1a8b WatchSource:0}: Error finding container 2e919e4c3e4359384731296d80612db41ca3e9d44b448f3fade1eabf01df1a8b: Status 404 returned error can't find the container with id 2e919e4c3e4359384731296d80612db41ca3e9d44b448f3fade1eabf01df1a8b Sep 10 15:57:26.548784 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:26.548754 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:26.548944 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:26.548814 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:26.553562 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:26.553542 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:27.154564 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:27.154520 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b88jf" event={"ID":"e1fc51db-e496-4b33-989b-e2b1e8c10890","Type":"ContainerStarted","Data":"2e919e4c3e4359384731296d80612db41ca3e9d44b448f3fade1eabf01df1a8b"} Sep 10 15:57:27.159947 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:27.159922 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:57:27.787878 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:57:27.787855 2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1fc51db_e496_4b33_989b_e2b1e8c10890.slice/crio-8baa33aaac06105608dd629f25dbf9ee7047f9042d14d713b8e75425b2c95de9.scope\": RecentStats: unable to find data in memory cache]" Sep 10 15:57:28.159548 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:28.159508 2576 generic.go:356] "Generic (PLEG): container finished" podID="e1fc51db-e496-4b33-989b-e2b1e8c10890" containerID="8baa33aaac06105608dd629f25dbf9ee7047f9042d14d713b8e75425b2c95de9" exitCode=0 Sep 10 15:57:28.159738 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:28.159600 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b88jf" event={"ID":"e1fc51db-e496-4b33-989b-e2b1e8c10890","Type":"ContainerDied","Data":"8baa33aaac06105608dd629f25dbf9ee7047f9042d14d713b8e75425b2c95de9"} Sep 10 15:57:29.165046 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:29.165012 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b88jf" event={"ID":"e1fc51db-e496-4b33-989b-e2b1e8c10890","Type":"ContainerStarted","Data":"630f9435010efc6441346d9f4a964291f9961d3565236fe568115e0b64def5e3"} Sep 10 15:57:29.165421 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:29.165063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b88jf" event={"ID":"e1fc51db-e496-4b33-989b-e2b1e8c10890","Type":"ContainerStarted","Data":"c16abc60068a1d35769b1761702fb33ada257577735709a520d0c2bc6ed7c23e"} Sep 10 15:57:29.186643 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:29.186603 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b88jf" podStartSLOduration=2.950716309 podStartE2EDuration="4.186589687s" totalImagesPullingTime="1.174417635s" totalInitContainerRuntime="61.455743ms" isStatefulPod=true podCreationTimestamp="2026-09-10 15:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:29.186284551 +0000 UTC m=+91.962342997" watchObservedRunningTime="2026-09-10 15:57:29.186589687 +0000 UTC m=+91.962648101" Sep 10 15:57:30.370762 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.370731 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-865c79d5f9-6ccl9"] Sep 10 15:57:30.373748 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.373732 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.383269 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.383243 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 15:57:30.391487 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.391456 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-865c79d5f9-6ccl9"] Sep 10 15:57:30.450527 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.450502 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-oauth-config\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.450660 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.450537 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-config\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.450660 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.450577 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-oauth-serving-cert\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.450660 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.450652 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-trusted-ca-bundle\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.450825 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.450682 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-service-ca\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.450825 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.450727 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-serving-cert\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.450825 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.450747 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxgxq\" (UniqueName: \"kubernetes.io/projected/a4dffaad-9d28-47ac-af11-df1f46f32c61-kube-api-access-lxgxq\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.551316 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.551286 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-oauth-config\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.551457 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.551324 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-config\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.551457 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.551358 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-oauth-serving-cert\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.551457 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.551404 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-trusted-ca-bundle\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.551457 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.551431 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-service-ca\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.551635 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.551566 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-serving-cert\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.551635 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.551603 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxgxq\" (UniqueName: \"kubernetes.io/projected/a4dffaad-9d28-47ac-af11-df1f46f32c61-kube-api-access-lxgxq\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.552244 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.552218 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-config\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.552351 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.552284 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-service-ca\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.552351 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.552293 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-oauth-serving-cert\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.552482 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.552466 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-trusted-ca-bundle\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.553789 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.553764 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-oauth-config\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.553981 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.553963 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-serving-cert\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.560040 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.560022 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxgxq\" (UniqueName: \"kubernetes.io/projected/a4dffaad-9d28-47ac-af11-df1f46f32c61-kube-api-access-lxgxq\") pod \"console-865c79d5f9-6ccl9\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.682253 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.682186 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:30.971768 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:30.971631 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-865c79d5f9-6ccl9"] Sep 10 15:57:30.974024 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:30.973999 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dffaad_9d28_47ac_af11_df1f46f32c61.slice/crio-29ebcc4f44a9d74693ab2c4b93c05778b8ce4a10125638e674dd41dd77de122b WatchSource:0}: Error finding container 29ebcc4f44a9d74693ab2c4b93c05778b8ce4a10125638e674dd41dd77de122b: Status 404 returned error can't find the container with id 29ebcc4f44a9d74693ab2c4b93c05778b8ce4a10125638e674dd41dd77de122b Sep 10 15:57:31.173360 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.173306 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-865c79d5f9-6ccl9" event={"ID":"a4dffaad-9d28-47ac-af11-df1f46f32c61","Type":"ContainerStarted","Data":"f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a"} Sep 10 15:57:31.173497 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.173376 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-865c79d5f9-6ccl9" event={"ID":"a4dffaad-9d28-47ac-af11-df1f46f32c61","Type":"ContainerStarted","Data":"29ebcc4f44a9d74693ab2c4b93c05778b8ce4a10125638e674dd41dd77de122b"} Sep 10 15:57:31.813186 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.813127 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-865c79d5f9-6ccl9" podStartSLOduration=1.8131112 podStartE2EDuration="1.8131112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:31.194575536 +0000 UTC m=+93.970633967" watchObservedRunningTime="2026-09-10 15:57:31.8131112 +0000 UTC m=+94.589169632" Sep 10 15:57:31.814365 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.814341 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 15:57:31.817925 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.817905 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.828026 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.828009 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 15:57:31.828382 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.828370 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 15:57:31.828582 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.828569 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 15:57:31.828648 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.828578 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 15:57:31.828736 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.828694 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 15:57:31.830036 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.830020 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-crpq0d4o76264\"" Sep 10 15:57:31.830351 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.830336 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 15:57:31.830807 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.830792 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 15:57:31.831194 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.831179 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-z8nd5\"" Sep 10 15:57:31.831429 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.831415 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 15:57:31.831596 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.831584 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 15:57:31.832309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.832295 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 15:57:31.832483 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.832469 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 15:57:31.833787 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.833772 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 15:57:31.835839 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.835823 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 15:57:31.837972 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.837953 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 15:57:31.839239 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.839223 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 15:57:31.845782 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.845764 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 15:57:31.962087 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962067 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-web-config\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962186 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962097 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962186 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962117 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962186 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962186 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962198 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962240 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962259 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962278 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962293 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962481 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962313 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962481 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962364 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962481 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962422 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config-out\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962481 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962448 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962481 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962465 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962621 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962490 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962621 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962516 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962621 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962532 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:31.962621 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:31.962552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-952s4\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-kube-api-access-952s4\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.063664 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.063610 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config-out\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.063664 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.063639 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.063664 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.063656 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.063827 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.063684 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.063896 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.063876 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.063948 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.063911 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.063948 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.063930 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-952s4\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-kube-api-access-952s4\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064042 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.063962 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-web-config\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064042 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.063993 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064042 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064022 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064196 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064081 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064196 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064119 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064196 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064147 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064196 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064179 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064380 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064202 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064380 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064234 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064380 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064266 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064380 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064293 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064380 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064327 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064380 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064353 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064742 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.064742 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.064467 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.065502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.065478 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.066696 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.066671 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config-out\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.067474 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.067014 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.067744 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.067475 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.067744 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.067597 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.068109 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.068080 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.068736 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.068127 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.068736 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.068399 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.068736 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.068507 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.068736 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.068541 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.068736 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.068579 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.069006 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.068985 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.069073 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.069053 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.069681 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.069653 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.069964 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.069945 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.070020 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.069966 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-web-config\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.070775 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.070756 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.075365 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.075338 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-952s4\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-kube-api-access-952s4\") pod \"prometheus-k8s-0\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.126404 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.126379 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:32.432577 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:32.431512 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 15:57:32.435902 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:32.435875 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a72908_21c3_40e3_bb75_b4e1bdc19509.slice/crio-0056ba163b891a836f6445da5e2173d8e3d44d8832babb92d3aba241f362acb2 WatchSource:0}: Error finding container 0056ba163b891a836f6445da5e2173d8e3d44d8832babb92d3aba241f362acb2: Status 404 returned error can't find the container with id 0056ba163b891a836f6445da5e2173d8e3d44d8832babb92d3aba241f362acb2 Sep 10 15:57:33.183023 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:33.182985 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerStarted","Data":"0056ba163b891a836f6445da5e2173d8e3d44d8832babb92d3aba241f362acb2"} Sep 10 15:57:34.187963 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:34.187931 2576 generic.go:356] "Generic (PLEG): container finished" podID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerID="15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f" exitCode=0 Sep 10 15:57:34.188298 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:34.188005 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerDied","Data":"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f"} Sep 10 15:57:36.623502 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.623461 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-865c79d5f9-6ccl9"] Sep 10 15:57:36.736475 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.736442 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d44ff6756-mk2tq"] Sep 10 15:57:36.740883 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.740855 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:36.824808 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.824779 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d44ff6756-mk2tq"] Sep 10 15:57:36.903148 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.903073 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-console-config\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:36.903148 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.903108 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-oauth-serving-cert\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:36.903148 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.903143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d47n7\" (UniqueName: \"kubernetes.io/projected/b4174fed-2576-441e-9dac-fea212857ae1-kube-api-access-d47n7\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:36.903359 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.903249 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-oauth-config\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:36.903359 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.903292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-serving-cert\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:36.903431 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.903359 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-trusted-ca-bundle\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:36.903431 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:36.903393 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-service-ca\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.004163 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.004124 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-trusted-ca-bundle\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.004343 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.004182 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-service-ca\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.004343 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.004210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-console-config\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.004343 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.004239 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-oauth-serving-cert\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.004343 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.004276 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d47n7\" (UniqueName: \"kubernetes.io/projected/b4174fed-2576-441e-9dac-fea212857ae1-kube-api-access-d47n7\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.004343 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.004331 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-oauth-config\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.004601 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.004380 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-serving-cert\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.005140 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.005110 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-trusted-ca-bundle\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.005240 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.005212 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-oauth-serving-cert\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.005364 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.005315 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-service-ca\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.005507 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.005444 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-console-config\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.007387 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.007364 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-oauth-config\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.007546 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.007526 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-serving-cert\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.018078 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.018048 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d47n7\" (UniqueName: \"kubernetes.io/projected/b4174fed-2576-441e-9dac-fea212857ae1-kube-api-access-d47n7\") pod \"console-6d44ff6756-mk2tq\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.051218 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.051194 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:37.966608 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:37.966575 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d44ff6756-mk2tq"] Sep 10 15:57:37.970790 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:57:37.970755 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4174fed_2576_441e_9dac_fea212857ae1.slice/crio-8ff7a2a75c63c562c9b9f614adc780f905edc0c59e3e09cb052ca17b12b986b5 WatchSource:0}: Error finding container 8ff7a2a75c63c562c9b9f614adc780f905edc0c59e3e09cb052ca17b12b986b5: Status 404 returned error can't find the container with id 8ff7a2a75c63c562c9b9f614adc780f905edc0c59e3e09cb052ca17b12b986b5 Sep 10 15:57:38.204169 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:38.204133 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerStarted","Data":"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca"} Sep 10 15:57:38.204301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:38.204205 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerStarted","Data":"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839"} Sep 10 15:57:38.205613 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:38.205587 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d44ff6756-mk2tq" event={"ID":"b4174fed-2576-441e-9dac-fea212857ae1","Type":"ContainerStarted","Data":"4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056"} Sep 10 15:57:38.205735 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:38.205632 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d44ff6756-mk2tq" event={"ID":"b4174fed-2576-441e-9dac-fea212857ae1","Type":"ContainerStarted","Data":"8ff7a2a75c63c562c9b9f614adc780f905edc0c59e3e09cb052ca17b12b986b5"} Sep 10 15:57:38.234309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:38.234223 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d44ff6756-mk2tq" podStartSLOduration=2.234208939 podStartE2EDuration="2.234208939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:38.233280639 +0000 UTC m=+101.009339062" watchObservedRunningTime="2026-09-10 15:57:38.234208939 +0000 UTC m=+101.010267370" Sep 10 15:57:40.217289 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:40.217258 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerStarted","Data":"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0"} Sep 10 15:57:40.217628 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:40.217324 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerStarted","Data":"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c"} Sep 10 15:57:40.217628 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:40.217351 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerStarted","Data":"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891"} Sep 10 15:57:40.217628 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:40.217379 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerStarted","Data":"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e"} Sep 10 15:57:40.258062 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:40.258000 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.636974324 podStartE2EDuration="9.25797949s" totalImagesPullingTime="6.598712496s" totalInitContainerRuntime="22.29267ms" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:40.256797477 +0000 UTC m=+103.032855910" watchObservedRunningTime="2026-09-10 15:57:40.25797949 +0000 UTC m=+103.034037923" Sep 10 15:57:40.682796 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:40.682699 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:57:42.126878 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:42.126846 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:57:46.116989 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:46.116960 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mwvrc" Sep 10 15:57:47.051564 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:47.051529 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:47.051744 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:47.051583 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:47.055901 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:47.055882 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:47.249554 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:47.249522 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:57:47.300028 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:57:47.299997 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f4d64bc46-ghgcn"] Sep 10 15:58:01.650704 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.650652 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-865c79d5f9-6ccl9" podUID="a4dffaad-9d28-47ac-af11-df1f46f32c61" containerName="console" containerID="cri-o://f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a" gracePeriod=15 Sep 10 15:58:01.883319 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.883296 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:58:01.989447 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.989414 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-oauth-serving-cert\") pod \"a4dffaad-9d28-47ac-af11-df1f46f32c61\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " Sep 10 15:58:01.989615 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.989452 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-trusted-ca-bundle\") pod \"a4dffaad-9d28-47ac-af11-df1f46f32c61\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " Sep 10 15:58:01.989615 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.989472 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-service-ca\") pod \"a4dffaad-9d28-47ac-af11-df1f46f32c61\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " Sep 10 15:58:01.989615 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.989511 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-config\") pod \"a4dffaad-9d28-47ac-af11-df1f46f32c61\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " Sep 10 15:58:01.989814 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.989658 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-oauth-config\") pod \"a4dffaad-9d28-47ac-af11-df1f46f32c61\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " Sep 10 15:58:01.989814 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.989699 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-serving-cert\") pod \"a4dffaad-9d28-47ac-af11-df1f46f32c61\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " Sep 10 15:58:01.989814 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.989757 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxgxq\" (UniqueName: \"kubernetes.io/projected/a4dffaad-9d28-47ac-af11-df1f46f32c61-kube-api-access-lxgxq\") pod \"a4dffaad-9d28-47ac-af11-df1f46f32c61\" (UID: \"a4dffaad-9d28-47ac-af11-df1f46f32c61\") " Sep 10 15:58:01.989964 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.989876 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a4dffaad-9d28-47ac-af11-df1f46f32c61" (UID: "a4dffaad-9d28-47ac-af11-df1f46f32c61"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:01.989964 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.989938 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-config" (OuterVolumeSpecName: "console-config") pod "a4dffaad-9d28-47ac-af11-df1f46f32c61" (UID: "a4dffaad-9d28-47ac-af11-df1f46f32c61"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:01.990077 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.990009 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a4dffaad-9d28-47ac-af11-df1f46f32c61" (UID: "a4dffaad-9d28-47ac-af11-df1f46f32c61"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:01.990077 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.990021 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-config\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:01.990077 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.990060 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-trusted-ca-bundle\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:01.990232 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.990192 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-service-ca" (OuterVolumeSpecName: "service-ca") pod "a4dffaad-9d28-47ac-af11-df1f46f32c61" (UID: "a4dffaad-9d28-47ac-af11-df1f46f32c61"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:01.991982 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.991958 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a4dffaad-9d28-47ac-af11-df1f46f32c61" (UID: "a4dffaad-9d28-47ac-af11-df1f46f32c61"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:01.992068 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.992006 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a4dffaad-9d28-47ac-af11-df1f46f32c61" (UID: "a4dffaad-9d28-47ac-af11-df1f46f32c61"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:01.992068 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:01.992049 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4dffaad-9d28-47ac-af11-df1f46f32c61-kube-api-access-lxgxq" (OuterVolumeSpecName: "kube-api-access-lxgxq") pod "a4dffaad-9d28-47ac-af11-df1f46f32c61" (UID: "a4dffaad-9d28-47ac-af11-df1f46f32c61"). InnerVolumeSpecName "kube-api-access-lxgxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:58:02.090738 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.090695 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-oauth-serving-cert\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:02.090738 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.090735 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dffaad-9d28-47ac-af11-df1f46f32c61-service-ca\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:02.090879 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.090745 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-oauth-config\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:02.090879 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.090754 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dffaad-9d28-47ac-af11-df1f46f32c61-console-serving-cert\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:02.090879 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.090763 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxgxq\" (UniqueName: \"kubernetes.io/projected/a4dffaad-9d28-47ac-af11-df1f46f32c61-kube-api-access-lxgxq\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:02.301653 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.301580 2576 generic.go:356] "Generic (PLEG): container finished" podID="a4dffaad-9d28-47ac-af11-df1f46f32c61" containerID="f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a" exitCode=0 Sep 10 15:58:02.301653 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.301632 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-865c79d5f9-6ccl9" Sep 10 15:58:02.301823 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.301666 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-865c79d5f9-6ccl9" event={"ID":"a4dffaad-9d28-47ac-af11-df1f46f32c61","Type":"ContainerDied","Data":"f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a"} Sep 10 15:58:02.301823 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.301739 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-865c79d5f9-6ccl9" event={"ID":"a4dffaad-9d28-47ac-af11-df1f46f32c61","Type":"ContainerDied","Data":"29ebcc4f44a9d74693ab2c4b93c05778b8ce4a10125638e674dd41dd77de122b"} Sep 10 15:58:02.301823 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.301765 2576 scope.go:117] "RemoveContainer" containerID="f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a" Sep 10 15:58:02.309951 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.309933 2576 scope.go:117] "RemoveContainer" containerID="f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a" Sep 10 15:58:02.310216 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:58:02.310196 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a\": container with ID starting with f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a not found: ID does not exist" containerID="f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a" Sep 10 15:58:02.310257 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.310227 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a"} err="failed to get container status \"f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a\": rpc error: code = NotFound desc = could not find container \"f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a\": container with ID starting with f9964642257478c802a77f788cb0e2cb0b41d1fd34ce6f00cb9eea759abd626a not found: ID does not exist" Sep 10 15:58:02.325230 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.325209 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-865c79d5f9-6ccl9"] Sep 10 15:58:02.331354 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:02.331327 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-865c79d5f9-6ccl9"] Sep 10 15:58:03.698346 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:03.698315 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4dffaad-9d28-47ac-af11-df1f46f32c61" path="/var/lib/kubelet/pods/a4dffaad-9d28-47ac-af11-df1f46f32c61/volumes" Sep 10 15:58:12.324333 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.324291 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f4d64bc46-ghgcn" podUID="4e8f8b2b-7866-4de8-986b-be57780d888e" containerName="console" containerID="cri-o://44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373" gracePeriod=15 Sep 10 15:58:12.553945 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.553919 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:58:12.566549 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.566525 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-oauth-serving-cert\") pod \"4e8f8b2b-7866-4de8-986b-be57780d888e\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " Sep 10 15:58:12.566662 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.566622 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-oauth-config\") pod \"4e8f8b2b-7866-4de8-986b-be57780d888e\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " Sep 10 15:58:12.566662 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.566651 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpgdn\" (UniqueName: \"kubernetes.io/projected/4e8f8b2b-7866-4de8-986b-be57780d888e-kube-api-access-bpgdn\") pod \"4e8f8b2b-7866-4de8-986b-be57780d888e\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " Sep 10 15:58:12.566802 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.566669 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-serving-cert\") pod \"4e8f8b2b-7866-4de8-986b-be57780d888e\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " Sep 10 15:58:12.566802 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.566690 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-console-config\") pod \"4e8f8b2b-7866-4de8-986b-be57780d888e\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " Sep 10 15:58:12.566904 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.566862 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-service-ca\") pod \"4e8f8b2b-7866-4de8-986b-be57780d888e\" (UID: \"4e8f8b2b-7866-4de8-986b-be57780d888e\") " Sep 10 15:58:12.566991 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.566963 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4e8f8b2b-7866-4de8-986b-be57780d888e" (UID: "4e8f8b2b-7866-4de8-986b-be57780d888e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:12.567148 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.567067 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-console-config" (OuterVolumeSpecName: "console-config") pod "4e8f8b2b-7866-4de8-986b-be57780d888e" (UID: "4e8f8b2b-7866-4de8-986b-be57780d888e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:12.567260 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.567117 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-oauth-serving-cert\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:12.567260 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.567204 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-service-ca" (OuterVolumeSpecName: "service-ca") pod "4e8f8b2b-7866-4de8-986b-be57780d888e" (UID: "4e8f8b2b-7866-4de8-986b-be57780d888e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:12.568824 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.568797 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4e8f8b2b-7866-4de8-986b-be57780d888e" (UID: "4e8f8b2b-7866-4de8-986b-be57780d888e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:12.568917 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.568858 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e8f8b2b-7866-4de8-986b-be57780d888e-kube-api-access-bpgdn" (OuterVolumeSpecName: "kube-api-access-bpgdn") pod "4e8f8b2b-7866-4de8-986b-be57780d888e" (UID: "4e8f8b2b-7866-4de8-986b-be57780d888e"). InnerVolumeSpecName "kube-api-access-bpgdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:58:12.568917 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.568863 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4e8f8b2b-7866-4de8-986b-be57780d888e" (UID: "4e8f8b2b-7866-4de8-986b-be57780d888e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:12.668521 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.668453 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-oauth-config\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:12.668521 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.668480 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpgdn\" (UniqueName: \"kubernetes.io/projected/4e8f8b2b-7866-4de8-986b-be57780d888e-kube-api-access-bpgdn\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:12.668521 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.668494 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e8f8b2b-7866-4de8-986b-be57780d888e-console-serving-cert\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:12.668521 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.668506 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-console-config\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:12.668521 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:12.668519 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8f8b2b-7866-4de8-986b-be57780d888e-service-ca\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:13.342402 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.342369 2576 generic.go:356] "Generic (PLEG): container finished" podID="4e8f8b2b-7866-4de8-986b-be57780d888e" containerID="44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373" exitCode=0 Sep 10 15:58:13.342831 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.342412 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4d64bc46-ghgcn" Sep 10 15:58:13.342831 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.342430 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4d64bc46-ghgcn" event={"ID":"4e8f8b2b-7866-4de8-986b-be57780d888e","Type":"ContainerDied","Data":"44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373"} Sep 10 15:58:13.342831 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.342470 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4d64bc46-ghgcn" event={"ID":"4e8f8b2b-7866-4de8-986b-be57780d888e","Type":"ContainerDied","Data":"9bd2904b51a8b6c4a4ae8e8f02fbc7bf9218946fb8f18cb76595719b3e2b67a0"} Sep 10 15:58:13.342831 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.342489 2576 scope.go:117] "RemoveContainer" containerID="44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373" Sep 10 15:58:13.351051 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.351033 2576 scope.go:117] "RemoveContainer" containerID="44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373" Sep 10 15:58:13.351300 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:58:13.351283 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373\": container with ID starting with 44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373 not found: ID does not exist" containerID="44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373" Sep 10 15:58:13.351345 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.351309 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373"} err="failed to get container status \"44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373\": rpc error: code = NotFound desc = could not find container \"44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373\": container with ID starting with 44f7db89c9c6131994ff8a3539090a65d037fb903d6f88ee3b925ec623e02373 not found: ID does not exist" Sep 10 15:58:13.363162 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.363136 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f4d64bc46-ghgcn"] Sep 10 15:58:13.367018 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.366999 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f4d64bc46-ghgcn"] Sep 10 15:58:13.701686 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:13.701655 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e8f8b2b-7866-4de8-986b-be57780d888e" path="/var/lib/kubelet/pods/4e8f8b2b-7866-4de8-986b-be57780d888e/volumes" Sep 10 15:58:32.127222 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:32.127178 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:32.148205 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:32.148177 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:32.421283 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:32.421210 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.133313 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.133227 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 15:58:50.133798 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.133613 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="prometheus" containerID="cri-o://cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" gracePeriod=600 Sep 10 15:58:50.133798 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.133643 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy" containerID="cri-o://68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" gracePeriod=600 Sep 10 15:58:50.133798 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.133666 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy-thanos" containerID="cri-o://4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" gracePeriod=600 Sep 10 15:58:50.133798 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.133663 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy-web" containerID="cri-o://5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" gracePeriod=600 Sep 10 15:58:50.133798 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.133735 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="thanos-sidecar" containerID="cri-o://33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" gracePeriod=600 Sep 10 15:58:50.133798 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.133677 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="config-reloader" containerID="cri-o://d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" gracePeriod=600 Sep 10 15:58:50.367627 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.367601 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.430651 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430583 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-1\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.430651 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430623 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config-out\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.430875 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430652 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-metrics-client-ca\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.430875 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430677 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-2\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.430875 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430722 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-metrics-client-certs\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.430875 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430751 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-kube-rbac-proxy\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.430875 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430782 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.430875 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430816 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-kubelet-serving-ca-bundle\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.430875 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430842 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-0\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431183 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430887 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-952s4\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-kube-api-access-952s4\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431183 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430918 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-thanos-prometheus-http-client-file\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431183 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430948 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-tls-assets\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431183 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430961 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:50.431183 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.430978 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-web-config\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431183 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431076 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-trusted-ca-bundle\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431183 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431108 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431183 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431142 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431183 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431173 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-db\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431619 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431205 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-grpc-tls\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431619 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431230 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-serving-certs-ca-bundle\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431619 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431266 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-tls\") pod \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\" (UID: \"c7a72908-21c3-40e3-bb75-b4e1bdc19509\") " Sep 10 15:58:50.431619 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431287 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:50.431619 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431472 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:50.431619 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431491 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.431619 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431508 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-metrics-client-ca\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.431996 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.431756 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:50.433016 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.432984 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:50.433134 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.433086 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:50.433368 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.433343 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:58:50.434197 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.434141 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 15:58:50.434391 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.434367 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config-out" (OuterVolumeSpecName: "config-out") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 15:58:50.434579 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.434550 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:50.434854 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.434818 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:50.434962 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.434919 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:50.435966 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.435915 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:50.436217 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.436177 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config" (OuterVolumeSpecName: "config") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:50.436217 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.436193 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:50.436450 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.436424 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:58:50.436849 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.436829 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-kube-api-access-952s4" (OuterVolumeSpecName: "kube-api-access-952s4") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "kube-api-access-952s4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:58:50.436949 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.436907 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:50.437275 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.437251 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:50.445520 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.445500 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-web-config" (OuterVolumeSpecName: "web-config") pod "c7a72908-21c3-40e3-bb75-b4e1bdc19509" (UID: "c7a72908-21c3-40e3-bb75-b4e1bdc19509"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:58:50.476028 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476003 2576 generic.go:356] "Generic (PLEG): container finished" podID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerID="4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" exitCode=0 Sep 10 15:58:50.476028 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476028 2576 generic.go:356] "Generic (PLEG): container finished" podID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerID="68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" exitCode=0 Sep 10 15:58:50.476136 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476035 2576 generic.go:356] "Generic (PLEG): container finished" podID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerID="5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" exitCode=0 Sep 10 15:58:50.476136 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476040 2576 generic.go:356] "Generic (PLEG): container finished" podID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerID="33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" exitCode=0 Sep 10 15:58:50.476136 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476045 2576 generic.go:356] "Generic (PLEG): container finished" podID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerID="d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" exitCode=0 Sep 10 15:58:50.476136 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476050 2576 generic.go:356] "Generic (PLEG): container finished" podID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerID="cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" exitCode=0 Sep 10 15:58:50.476136 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476077 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.476136 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476085 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerDied","Data":"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0"} Sep 10 15:58:50.476309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476180 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerDied","Data":"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c"} Sep 10 15:58:50.476309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476209 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerDied","Data":"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891"} Sep 10 15:58:50.476309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476239 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerDied","Data":"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e"} Sep 10 15:58:50.476309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476247 2576 scope.go:117] "RemoveContainer" containerID="4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" Sep 10 15:58:50.476309 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476280 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerDied","Data":"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca"} Sep 10 15:58:50.476526 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476321 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerDied","Data":"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839"} Sep 10 15:58:50.476526 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.476363 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c7a72908-21c3-40e3-bb75-b4e1bdc19509","Type":"ContainerDied","Data":"0056ba163b891a836f6445da5e2173d8e3d44d8832babb92d3aba241f362acb2"} Sep 10 15:58:50.483500 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.483482 2576 scope.go:117] "RemoveContainer" containerID="68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" Sep 10 15:58:50.490647 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.490630 2576 scope.go:117] "RemoveContainer" containerID="5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" Sep 10 15:58:50.496662 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.496647 2576 scope.go:117] "RemoveContainer" containerID="33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" Sep 10 15:58:50.501132 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.501110 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 15:58:50.503382 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.503367 2576 scope.go:117] "RemoveContainer" containerID="d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" Sep 10 15:58:50.505780 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.505759 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 15:58:50.509623 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.509594 2576 scope.go:117] "RemoveContainer" containerID="cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" Sep 10 15:58:50.517414 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.517396 2576 scope.go:117] "RemoveContainer" containerID="15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f" Sep 10 15:58:50.523797 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.523771 2576 scope.go:117] "RemoveContainer" containerID="4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" Sep 10 15:58:50.524044 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:58:50.524025 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": container with ID starting with 4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0 not found: ID does not exist" containerID="4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" Sep 10 15:58:50.524097 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.524054 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0"} err="failed to get container status \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": rpc error: code = NotFound desc = could not find container \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": container with ID starting with 4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0 not found: ID does not exist" Sep 10 15:58:50.524097 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.524074 2576 scope.go:117] "RemoveContainer" containerID="68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" Sep 10 15:58:50.524290 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:58:50.524273 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": container with ID starting with 68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c not found: ID does not exist" containerID="68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" Sep 10 15:58:50.524375 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.524295 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c"} err="failed to get container status \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": rpc error: code = NotFound desc = could not find container \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": container with ID starting with 68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c not found: ID does not exist" Sep 10 15:58:50.524375 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.524311 2576 scope.go:117] "RemoveContainer" containerID="5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" Sep 10 15:58:50.524517 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:58:50.524503 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": container with ID starting with 5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891 not found: ID does not exist" containerID="5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" Sep 10 15:58:50.524551 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.524522 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891"} err="failed to get container status \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": rpc error: code = NotFound desc = could not find container \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": container with ID starting with 5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891 not found: ID does not exist" Sep 10 15:58:50.524551 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.524533 2576 scope.go:117] "RemoveContainer" containerID="33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" Sep 10 15:58:50.524820 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:58:50.524804 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": container with ID starting with 33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e not found: ID does not exist" containerID="33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" Sep 10 15:58:50.524885 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.524823 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e"} err="failed to get container status \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": rpc error: code = NotFound desc = could not find container \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": container with ID starting with 33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e not found: ID does not exist" Sep 10 15:58:50.524885 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.524835 2576 scope.go:117] "RemoveContainer" containerID="d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" Sep 10 15:58:50.525031 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:58:50.525017 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": container with ID starting with d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca not found: ID does not exist" containerID="d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" Sep 10 15:58:50.525067 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525035 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca"} err="failed to get container status \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": rpc error: code = NotFound desc = could not find container \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": container with ID starting with d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca not found: ID does not exist" Sep 10 15:58:50.525067 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525047 2576 scope.go:117] "RemoveContainer" containerID="cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" Sep 10 15:58:50.525228 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:58:50.525215 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": container with ID starting with cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839 not found: ID does not exist" containerID="cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" Sep 10 15:58:50.525266 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525230 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839"} err="failed to get container status \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": rpc error: code = NotFound desc = could not find container \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": container with ID starting with cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839 not found: ID does not exist" Sep 10 15:58:50.525266 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525241 2576 scope.go:117] "RemoveContainer" containerID="15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f" Sep 10 15:58:50.525420 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:58:50.525405 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": container with ID starting with 15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f not found: ID does not exist" containerID="15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f" Sep 10 15:58:50.525461 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525422 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f"} err="failed to get container status \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": rpc error: code = NotFound desc = could not find container \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": container with ID starting with 15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f not found: ID does not exist" Sep 10 15:58:50.525461 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525432 2576 scope.go:117] "RemoveContainer" containerID="4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" Sep 10 15:58:50.525651 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525635 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0"} err="failed to get container status \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": rpc error: code = NotFound desc = could not find container \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": container with ID starting with 4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0 not found: ID does not exist" Sep 10 15:58:50.525695 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525652 2576 scope.go:117] "RemoveContainer" containerID="68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" Sep 10 15:58:50.525932 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525914 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c"} err="failed to get container status \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": rpc error: code = NotFound desc = could not find container \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": container with ID starting with 68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c not found: ID does not exist" Sep 10 15:58:50.526024 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.525933 2576 scope.go:117] "RemoveContainer" containerID="5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" Sep 10 15:58:50.526165 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.526147 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891"} err="failed to get container status \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": rpc error: code = NotFound desc = could not find container \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": container with ID starting with 5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891 not found: ID does not exist" Sep 10 15:58:50.526232 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.526165 2576 scope.go:117] "RemoveContainer" containerID="33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" Sep 10 15:58:50.526384 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.526368 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e"} err="failed to get container status \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": rpc error: code = NotFound desc = could not find container \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": container with ID starting with 33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e not found: ID does not exist" Sep 10 15:58:50.526452 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.526385 2576 scope.go:117] "RemoveContainer" containerID="d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" Sep 10 15:58:50.526624 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.526607 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca"} err="failed to get container status \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": rpc error: code = NotFound desc = could not find container \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": container with ID starting with d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca not found: ID does not exist" Sep 10 15:58:50.526670 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.526625 2576 scope.go:117] "RemoveContainer" containerID="cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" Sep 10 15:58:50.526845 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.526822 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839"} err="failed to get container status \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": rpc error: code = NotFound desc = could not find container \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": container with ID starting with cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839 not found: ID does not exist" Sep 10 15:58:50.526896 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.526846 2576 scope.go:117] "RemoveContainer" containerID="15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f" Sep 10 15:58:50.527062 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527045 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f"} err="failed to get container status \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": rpc error: code = NotFound desc = could not find container \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": container with ID starting with 15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f not found: ID does not exist" Sep 10 15:58:50.527129 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527063 2576 scope.go:117] "RemoveContainer" containerID="4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" Sep 10 15:58:50.527288 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527270 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0"} err="failed to get container status \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": rpc error: code = NotFound desc = could not find container \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": container with ID starting with 4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0 not found: ID does not exist" Sep 10 15:58:50.527335 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527289 2576 scope.go:117] "RemoveContainer" containerID="68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" Sep 10 15:58:50.527480 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527462 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c"} err="failed to get container status \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": rpc error: code = NotFound desc = could not find container \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": container with ID starting with 68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c not found: ID does not exist" Sep 10 15:58:50.527543 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527480 2576 scope.go:117] "RemoveContainer" containerID="5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" Sep 10 15:58:50.527719 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527687 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891"} err="failed to get container status \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": rpc error: code = NotFound desc = could not find container \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": container with ID starting with 5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891 not found: ID does not exist" Sep 10 15:58:50.527768 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527721 2576 scope.go:117] "RemoveContainer" containerID="33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" Sep 10 15:58:50.527910 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527892 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e"} err="failed to get container status \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": rpc error: code = NotFound desc = could not find container \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": container with ID starting with 33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e not found: ID does not exist" Sep 10 15:58:50.527974 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.527910 2576 scope.go:117] "RemoveContainer" containerID="d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" Sep 10 15:58:50.528146 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.528129 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca"} err="failed to get container status \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": rpc error: code = NotFound desc = could not find container \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": container with ID starting with d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca not found: ID does not exist" Sep 10 15:58:50.528194 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.528147 2576 scope.go:117] "RemoveContainer" containerID="cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" Sep 10 15:58:50.528347 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.528331 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839"} err="failed to get container status \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": rpc error: code = NotFound desc = could not find container \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": container with ID starting with cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839 not found: ID does not exist" Sep 10 15:58:50.528408 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.528349 2576 scope.go:117] "RemoveContainer" containerID="15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f" Sep 10 15:58:50.528555 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.528540 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f"} err="failed to get container status \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": rpc error: code = NotFound desc = could not find container \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": container with ID starting with 15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f not found: ID does not exist" Sep 10 15:58:50.528614 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.528556 2576 scope.go:117] "RemoveContainer" containerID="4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" Sep 10 15:58:50.528792 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.528777 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0"} err="failed to get container status \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": rpc error: code = NotFound desc = could not find container \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": container with ID starting with 4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0 not found: ID does not exist" Sep 10 15:58:50.528792 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.528792 2576 scope.go:117] "RemoveContainer" containerID="68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" Sep 10 15:58:50.529020 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.529001 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c"} err="failed to get container status \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": rpc error: code = NotFound desc = could not find container \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": container with ID starting with 68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c not found: ID does not exist" Sep 10 15:58:50.529070 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.529020 2576 scope.go:117] "RemoveContainer" containerID="5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" Sep 10 15:58:50.529249 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.529232 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891"} err="failed to get container status \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": rpc error: code = NotFound desc = could not find container \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": container with ID starting with 5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891 not found: ID does not exist" Sep 10 15:58:50.529296 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.529249 2576 scope.go:117] "RemoveContainer" containerID="33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" Sep 10 15:58:50.529537 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.529488 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e"} err="failed to get container status \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": rpc error: code = NotFound desc = could not find container \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": container with ID starting with 33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e not found: ID does not exist" Sep 10 15:58:50.529646 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.529540 2576 scope.go:117] "RemoveContainer" containerID="d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" Sep 10 15:58:50.530039 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.529975 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca"} err="failed to get container status \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": rpc error: code = NotFound desc = could not find container \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": container with ID starting with d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca not found: ID does not exist" Sep 10 15:58:50.530039 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.530005 2576 scope.go:117] "RemoveContainer" containerID="cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" Sep 10 15:58:50.530344 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.530316 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839"} err="failed to get container status \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": rpc error: code = NotFound desc = could not find container \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": container with ID starting with cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839 not found: ID does not exist" Sep 10 15:58:50.530424 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.530346 2576 scope.go:117] "RemoveContainer" containerID="15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f" Sep 10 15:58:50.530657 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.530635 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f"} err="failed to get container status \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": rpc error: code = NotFound desc = could not find container \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": container with ID starting with 15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f not found: ID does not exist" Sep 10 15:58:50.530731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.530660 2576 scope.go:117] "RemoveContainer" containerID="4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" Sep 10 15:58:50.530968 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.530948 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0"} err="failed to get container status \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": rpc error: code = NotFound desc = could not find container \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": container with ID starting with 4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0 not found: ID does not exist" Sep 10 15:58:50.531063 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.530969 2576 scope.go:117] "RemoveContainer" containerID="68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" Sep 10 15:58:50.531280 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531262 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c"} err="failed to get container status \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": rpc error: code = NotFound desc = could not find container \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": container with ID starting with 68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c not found: ID does not exist" Sep 10 15:58:50.531334 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531283 2576 scope.go:117] "RemoveContainer" containerID="5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" Sep 10 15:58:50.531368 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531272 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 15:58:50.531569 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531558 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4dffaad-9d28-47ac-af11-df1f46f32c61" containerName="console" Sep 10 15:58:50.531605 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531571 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4dffaad-9d28-47ac-af11-df1f46f32c61" containerName="console" Sep 10 15:58:50.531605 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531572 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891"} err="failed to get container status \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": rpc error: code = NotFound desc = could not find container \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": container with ID starting with 5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891 not found: ID does not exist" Sep 10 15:58:50.531605 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531588 2576 scope.go:117] "RemoveContainer" containerID="33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531581 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531635 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531651 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="prometheus" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531656 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="prometheus" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531665 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="config-reloader" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531671 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="config-reloader" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531679 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="thanos-sidecar" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531685 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="thanos-sidecar" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531691 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy-thanos" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531696 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy-thanos" Sep 10 15:58:50.531731 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531734 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e8f8b2b-7866-4de8-986b-be57780d888e" containerName="console" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531744 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e8f8b2b-7866-4de8-986b-be57780d888e" containerName="console" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531755 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="init-config-reloader" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531763 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="init-config-reloader" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531773 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy-web" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531781 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy-web" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531818 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531833 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531844 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531850 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="thanos-sidecar" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531852 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-952s4\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-kube-api-access-952s4\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531858 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531862 2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531872 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7a72908-21c3-40e3-bb75-b4e1bdc19509-tls-assets\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531895 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e"} err="failed to get container status \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": rpc error: code = NotFound desc = could not find container \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": container with ID starting with 33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e not found: ID does not exist" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531910 2576 scope.go:117] "RemoveContainer" containerID="d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531902 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-web-config\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531865 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy-thanos" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531968 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4dffaad-9d28-47ac-af11-df1f46f32c61" containerName="console" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531979 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="kube-rbac-proxy-web" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531990 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e8f8b2b-7866-4de8-986b-be57780d888e" containerName="console" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532000 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="config-reloader" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532011 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" containerName="prometheus" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.531944 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532041 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532069 2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532082 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-db\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.532211 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532094 2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-grpc-tls\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532103 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532113 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532121 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7a72908-21c3-40e3-bb75-b4e1bdc19509-config-out\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532137 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c7a72908-21c3-40e3-bb75-b4e1bdc19509-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532148 2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-metrics-client-certs\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532161 2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7a72908-21c3-40e3-bb75-b4e1bdc19509-secret-kube-rbac-proxy\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532143 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca"} err="failed to get container status \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": rpc error: code = NotFound desc = could not find container \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": container with ID starting with d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca not found: ID does not exist" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532175 2576 scope.go:117] "RemoveContainer" containerID="cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532391 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839"} err="failed to get container status \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": rpc error: code = NotFound desc = could not find container \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": container with ID starting with cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839 not found: ID does not exist" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532407 2576 scope.go:117] "RemoveContainer" containerID="15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532634 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f"} err="failed to get container status \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": rpc error: code = NotFound desc = could not find container \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": container with ID starting with 15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f not found: ID does not exist" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532650 2576 scope.go:117] "RemoveContainer" containerID="4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532893 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0"} err="failed to get container status \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": rpc error: code = NotFound desc = could not find container \"4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0\": container with ID starting with 4be4edc0f8512e9e1630d2119a0e8a4b672a4a4810c85df2da00a6747754a2e0 not found: ID does not exist" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.532907 2576 scope.go:117] "RemoveContainer" containerID="68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.533090 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c"} err="failed to get container status \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": rpc error: code = NotFound desc = could not find container \"68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c\": container with ID starting with 68ff0daa13f284b451227760f48b6e4ef6c4dcb42f0bccbf4aac42776892037c not found: ID does not exist" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.533110 2576 scope.go:117] "RemoveContainer" containerID="5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.533306 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891"} err="failed to get container status \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": rpc error: code = NotFound desc = could not find container \"5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891\": container with ID starting with 5ba4ee3d83ba75125b4f14de2d4eced550c8d9b800ae624de0adf94a38a14891 not found: ID does not exist" Sep 10 15:58:50.533406 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.533318 2576 scope.go:117] "RemoveContainer" containerID="33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e" Sep 10 15:58:50.534075 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.533548 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e"} err="failed to get container status \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": rpc error: code = NotFound desc = could not find container \"33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e\": container with ID starting with 33339e50a9e469c1532ca9dac146f7557decf019fcac072c497a3848b3d88c8e not found: ID does not exist" Sep 10 15:58:50.534075 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.533566 2576 scope.go:117] "RemoveContainer" containerID="d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca" Sep 10 15:58:50.534075 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.533802 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca"} err="failed to get container status \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": rpc error: code = NotFound desc = could not find container \"d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca\": container with ID starting with d38ee9f8118431f8259e0d8a841d05911c48378b488716024a3aa9fe3bc240ca not found: ID does not exist" Sep 10 15:58:50.534075 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.533827 2576 scope.go:117] "RemoveContainer" containerID="cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839" Sep 10 15:58:50.534075 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.534029 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839"} err="failed to get container status \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": rpc error: code = NotFound desc = could not find container \"cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839\": container with ID starting with cab3c3da9b4641837a320864874d25eecae9ccc335c0419fee0ef255e1217839 not found: ID does not exist" Sep 10 15:58:50.534075 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.534043 2576 scope.go:117] "RemoveContainer" containerID="15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f" Sep 10 15:58:50.534301 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.534231 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f"} err="failed to get container status \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": rpc error: code = NotFound desc = could not find container \"15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f\": container with ID starting with 15a8be99edf97ca2fbca3cef2eb05ceb0cffd41ce010e679c795a785ee23348f not found: ID does not exist" Sep 10 15:58:50.537136 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.537121 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.539949 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.539930 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 15:58:50.540157 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540131 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 15:58:50.540354 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540183 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 15:58:50.540354 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540247 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 15:58:50.540354 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540324 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 15:58:50.540606 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540466 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 15:58:50.540606 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540542 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 15:58:50.540606 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540569 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 15:58:50.540606 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540575 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 15:58:50.540821 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540660 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-crpq0d4o76264\"" Sep 10 15:58:50.540905 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540889 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 15:58:50.540965 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540905 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-z8nd5\"" Sep 10 15:58:50.541012 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.540986 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 15:58:50.541111 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.541076 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 15:58:50.541285 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.541262 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 15:58:50.543864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.543543 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 15:58:50.546037 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.546019 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 15:58:50.551988 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.551968 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 15:58:50.632858 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.632825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.632993 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.632881 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-web-config\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.632993 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.632899 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.632993 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.632942 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.632993 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.632975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633127 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633005 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633127 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633020 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-config-out\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633127 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633040 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633127 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633067 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l57wz\" (UniqueName: \"kubernetes.io/projected/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-kube-api-access-l57wz\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633127 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633127 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633105 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633291 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633138 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-config\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633291 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633291 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633207 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633291 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633227 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633291 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633246 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633291 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633287 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633462 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633305 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633462 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633326 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.633462 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.633340 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.733932 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.733911 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734036 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.733937 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-config-out\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734036 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.733955 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734036 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.733970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l57wz\" (UniqueName: \"kubernetes.io/projected/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-kube-api-access-l57wz\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734036 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.733996 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734036 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734021 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734357 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734046 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-config\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734357 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734069 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734357 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734101 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734357 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734129 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734357 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734155 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734357 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734185 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734643 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734376 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734643 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734416 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734643 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734621 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734815 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734666 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734815 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734699 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-web-config\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734815 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734763 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.734815 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734785 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.735011 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.734817 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.736242 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.735277 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.736242 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.735327 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.736242 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.735911 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.737043 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.736738 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-config-out\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.737146 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.737124 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.737146 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.737134 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.737274 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.737250 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.737362 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.737341 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.737419 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.737395 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.737606 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.737586 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.737695 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.737675 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.738288 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.738262 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.738525 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.738502 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.738810 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.738790 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.739305 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.739286 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.739695 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.739671 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.739856 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.739837 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.739988 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.739973 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-config\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.740025 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.739985 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-web-config\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.742289 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.742272 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l57wz\" (UniqueName: \"kubernetes.io/projected/40be2d3d-5c49-4ebf-bc91-65bffd1a898c-kube-api-access-l57wz\") pod \"prometheus-k8s-0\" (UID: \"40be2d3d-5c49-4ebf-bc91-65bffd1a898c\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:50.847003 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:50.846971 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:58:51.126143 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:51.126118 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 15:58:51.128362 ip-10-0-129-74 kubenswrapper[2576]: W0910 15:58:51.128333 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40be2d3d_5c49_4ebf_bc91_65bffd1a898c.slice/crio-af7079489261865d4c5ed1342ddc473b339908f53ea80c87c8c17d1cd954797b WatchSource:0}: Error finding container af7079489261865d4c5ed1342ddc473b339908f53ea80c87c8c17d1cd954797b: Status 404 returned error can't find the container with id af7079489261865d4c5ed1342ddc473b339908f53ea80c87c8c17d1cd954797b Sep 10 15:58:51.481722 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:51.481681 2576 generic.go:356] "Generic (PLEG): container finished" podID="40be2d3d-5c49-4ebf-bc91-65bffd1a898c" containerID="f9f058f0bea4d7da1f11a9a445d7475ea2927e74ba92901836a4c072da601411" exitCode=0 Sep 10 15:58:51.482073 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:51.481746 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40be2d3d-5c49-4ebf-bc91-65bffd1a898c","Type":"ContainerDied","Data":"f9f058f0bea4d7da1f11a9a445d7475ea2927e74ba92901836a4c072da601411"} Sep 10 15:58:51.482073 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:51.481840 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40be2d3d-5c49-4ebf-bc91-65bffd1a898c","Type":"ContainerStarted","Data":"af7079489261865d4c5ed1342ddc473b339908f53ea80c87c8c17d1cd954797b"} Sep 10 15:58:51.703013 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:51.702673 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7a72908-21c3-40e3-bb75-b4e1bdc19509" path="/var/lib/kubelet/pods/c7a72908-21c3-40e3-bb75-b4e1bdc19509/volumes" Sep 10 15:58:52.489248 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:52.489217 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40be2d3d-5c49-4ebf-bc91-65bffd1a898c","Type":"ContainerStarted","Data":"73c05b7960fa4fa8fc8d06c747c9fbbc40be517d5f7e5c3f65ca223c93602a50"} Sep 10 15:58:52.489632 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:52.489281 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40be2d3d-5c49-4ebf-bc91-65bffd1a898c","Type":"ContainerStarted","Data":"6275359d8fa207541a2056bc8ad6bf9ad3fac23b9fee1264e389773394b83e9a"} Sep 10 15:58:52.489632 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:52.489306 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40be2d3d-5c49-4ebf-bc91-65bffd1a898c","Type":"ContainerStarted","Data":"81f4985bc16f0412be067fe839ff7064be8c9261e3b7e4a255ffb7745e517500"} Sep 10 15:58:52.489632 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:52.489331 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40be2d3d-5c49-4ebf-bc91-65bffd1a898c","Type":"ContainerStarted","Data":"98e4f1e01838d430d4a9f21f0070483d11eb454a58f966258b6fa2fd0ef95b5d"} Sep 10 15:58:52.489632 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:52.489354 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40be2d3d-5c49-4ebf-bc91-65bffd1a898c","Type":"ContainerStarted","Data":"5e66f443830611b3607326ff4c4ec62b32a1c77a7535422f212e02a187e11c92"} Sep 10 15:58:52.489632 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:52.489376 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"40be2d3d-5c49-4ebf-bc91-65bffd1a898c","Type":"ContainerStarted","Data":"7b2d8911494ee53be9558fb860ec5ca04481ed7cfd1e7b73bd56e949e31e64e9"} Sep 10 15:58:52.518457 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:52.518390 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.498854308 podStartE2EDuration="2.518373036s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.518728ms" isStatefulPod=false podCreationTimestamp="2026-09-10 15:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:58:52.516047215 +0000 UTC m=+175.292105647" watchObservedRunningTime="2026-09-10 15:58:52.518373036 +0000 UTC m=+175.294431469" Sep 10 15:58:55.801419 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:55.801387 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d44ff6756-mk2tq"] Sep 10 15:58:55.847274 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:58:55.847243 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:59:20.822635 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:20.822592 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d44ff6756-mk2tq" podUID="b4174fed-2576-441e-9dac-fea212857ae1" containerName="console" containerID="cri-o://4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056" gracePeriod=15 Sep 10 15:59:21.059384 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.059360 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:59:21.140451 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140381 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-oauth-config\") pod \"b4174fed-2576-441e-9dac-fea212857ae1\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " Sep 10 15:59:21.140451 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140424 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-service-ca\") pod \"b4174fed-2576-441e-9dac-fea212857ae1\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " Sep 10 15:59:21.140451 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140443 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-trusted-ca-bundle\") pod \"b4174fed-2576-441e-9dac-fea212857ae1\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " Sep 10 15:59:21.140742 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140479 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-console-config\") pod \"b4174fed-2576-441e-9dac-fea212857ae1\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " Sep 10 15:59:21.140742 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140509 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d47n7\" (UniqueName: \"kubernetes.io/projected/b4174fed-2576-441e-9dac-fea212857ae1-kube-api-access-d47n7\") pod \"b4174fed-2576-441e-9dac-fea212857ae1\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " Sep 10 15:59:21.140742 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140696 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-oauth-serving-cert\") pod \"b4174fed-2576-441e-9dac-fea212857ae1\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " Sep 10 15:59:21.140908 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140759 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-serving-cert\") pod \"b4174fed-2576-441e-9dac-fea212857ae1\" (UID: \"b4174fed-2576-441e-9dac-fea212857ae1\") " Sep 10 15:59:21.140966 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140943 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-service-ca" (OuterVolumeSpecName: "service-ca") pod "b4174fed-2576-441e-9dac-fea212857ae1" (UID: "b4174fed-2576-441e-9dac-fea212857ae1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:59:21.141007 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140959 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-console-config" (OuterVolumeSpecName: "console-config") pod "b4174fed-2576-441e-9dac-fea212857ae1" (UID: "b4174fed-2576-441e-9dac-fea212857ae1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:59:21.141007 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.140969 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b4174fed-2576-441e-9dac-fea212857ae1" (UID: "b4174fed-2576-441e-9dac-fea212857ae1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:59:21.141077 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.141059 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b4174fed-2576-441e-9dac-fea212857ae1" (UID: "b4174fed-2576-441e-9dac-fea212857ae1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:59:21.142498 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.142476 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b4174fed-2576-441e-9dac-fea212857ae1" (UID: "b4174fed-2576-441e-9dac-fea212857ae1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:59:21.142695 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.142674 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4174fed-2576-441e-9dac-fea212857ae1-kube-api-access-d47n7" (OuterVolumeSpecName: "kube-api-access-d47n7") pod "b4174fed-2576-441e-9dac-fea212857ae1" (UID: "b4174fed-2576-441e-9dac-fea212857ae1"). InnerVolumeSpecName "kube-api-access-d47n7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:59:21.142828 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.142811 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b4174fed-2576-441e-9dac-fea212857ae1" (UID: "b4174fed-2576-441e-9dac-fea212857ae1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:59:21.241410 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.241383 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d47n7\" (UniqueName: \"kubernetes.io/projected/b4174fed-2576-441e-9dac-fea212857ae1-kube-api-access-d47n7\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:59:21.241410 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.241409 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-oauth-serving-cert\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:59:21.241558 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.241423 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-serving-cert\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:59:21.241558 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.241434 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4174fed-2576-441e-9dac-fea212857ae1-console-oauth-config\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:59:21.241558 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.241446 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-service-ca\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:59:21.241558 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.241459 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-trusted-ca-bundle\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:59:21.241558 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.241471 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4174fed-2576-441e-9dac-fea212857ae1-console-config\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 15:59:21.592200 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.592167 2576 generic.go:356] "Generic (PLEG): container finished" podID="b4174fed-2576-441e-9dac-fea212857ae1" containerID="4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056" exitCode=0 Sep 10 15:59:21.592354 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.592225 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d44ff6756-mk2tq" Sep 10 15:59:21.592354 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.592228 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d44ff6756-mk2tq" event={"ID":"b4174fed-2576-441e-9dac-fea212857ae1","Type":"ContainerDied","Data":"4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056"} Sep 10 15:59:21.592354 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.592318 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d44ff6756-mk2tq" event={"ID":"b4174fed-2576-441e-9dac-fea212857ae1","Type":"ContainerDied","Data":"8ff7a2a75c63c562c9b9f614adc780f905edc0c59e3e09cb052ca17b12b986b5"} Sep 10 15:59:21.592354 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.592347 2576 scope.go:117] "RemoveContainer" containerID="4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056" Sep 10 15:59:21.600509 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.600492 2576 scope.go:117] "RemoveContainer" containerID="4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056" Sep 10 15:59:21.600803 ip-10-0-129-74 kubenswrapper[2576]: E0910 15:59:21.600784 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056\": container with ID starting with 4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056 not found: ID does not exist" containerID="4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056" Sep 10 15:59:21.600864 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.600810 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056"} err="failed to get container status \"4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056\": rpc error: code = NotFound desc = could not find container \"4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056\": container with ID starting with 4f83c1b446a1bbef7d872ae84a757d9ac5447c089bfda1be1683edc10f9f5056 not found: ID does not exist" Sep 10 15:59:21.612417 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.612398 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d44ff6756-mk2tq"] Sep 10 15:59:21.618356 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.618337 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d44ff6756-mk2tq"] Sep 10 15:59:21.698456 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:21.698431 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4174fed-2576-441e-9dac-fea212857ae1" path="/var/lib/kubelet/pods/b4174fed-2576-441e-9dac-fea212857ae1/volumes" Sep 10 15:59:50.848151 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:50.848115 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:59:50.863317 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:50.863292 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 15:59:51.714566 ip-10-0-129-74 kubenswrapper[2576]: I0910 15:59:51.714536 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 16:00:38.507530 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.507495 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p"] Sep 10 16:00:38.507943 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.507757 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4174fed-2576-441e-9dac-fea212857ae1" containerName="console" Sep 10 16:00:38.507943 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.507770 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4174fed-2576-441e-9dac-fea212857ae1" containerName="console" Sep 10 16:00:38.507943 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.507848 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4174fed-2576-441e-9dac-fea212857ae1" containerName="console" Sep 10 16:00:38.510756 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.510741 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.513430 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.513408 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 16:00:38.514115 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.514095 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 16:00:38.514877 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.514857 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 16:00:38.515244 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.515228 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sfvsh\"" Sep 10 16:00:38.515334 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.515321 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 16:00:38.531101 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.531081 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p"] Sep 10 16:00:38.565593 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.565569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d7009d60-9919-4e28-a5e9-297b09620e31-manifests\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.565736 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.565598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7009d60-9919-4e28-a5e9-297b09620e31-tmp\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.565736 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.565620 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7009d60-9919-4e28-a5e9-297b09620e31-webhook-cert\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.565736 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.565640 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxqks\" (UniqueName: \"kubernetes.io/projected/d7009d60-9919-4e28-a5e9-297b09620e31-kube-api-access-zxqks\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.565886 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.565764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d7009d60-9919-4e28-a5e9-297b09620e31-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.666446 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.666425 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d7009d60-9919-4e28-a5e9-297b09620e31-manifests\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.666546 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.666453 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7009d60-9919-4e28-a5e9-297b09620e31-tmp\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.666546 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.666475 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7009d60-9919-4e28-a5e9-297b09620e31-webhook-cert\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.666546 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.666493 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxqks\" (UniqueName: \"kubernetes.io/projected/d7009d60-9919-4e28-a5e9-297b09620e31-kube-api-access-zxqks\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.666546 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.666517 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d7009d60-9919-4e28-a5e9-297b09620e31-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.666878 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.666860 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d7009d60-9919-4e28-a5e9-297b09620e31-manifests\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.668703 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.668677 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7009d60-9919-4e28-a5e9-297b09620e31-tmp\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.668936 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.668917 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d7009d60-9919-4e28-a5e9-297b09620e31-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.669039 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.669021 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7009d60-9919-4e28-a5e9-297b09620e31-webhook-cert\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.679221 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.679197 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxqks\" (UniqueName: \"kubernetes.io/projected/d7009d60-9919-4e28-a5e9-297b09620e31-kube-api-access-zxqks\") pod \"opendatahub-operator-controller-manager-5995b6f449-65b7p\" (UID: \"d7009d60-9919-4e28-a5e9-297b09620e31\") " pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:38.820631 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:38.820561 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:39.103549 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:39.103470 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p"] Sep 10 16:00:39.109334 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:00:39.109308 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7009d60_9919_4e28_a5e9_297b09620e31.slice/crio-1e696ef7e374e82f10f3836c292176c3d08fcb41aaebfbb8c43cc16eb63f0e72 WatchSource:0}: Error finding container 1e696ef7e374e82f10f3836c292176c3d08fcb41aaebfbb8c43cc16eb63f0e72: Status 404 returned error can't find the container with id 1e696ef7e374e82f10f3836c292176c3d08fcb41aaebfbb8c43cc16eb63f0e72 Sep 10 16:00:39.849550 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:39.849506 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" event={"ID":"d7009d60-9919-4e28-a5e9-297b09620e31","Type":"ContainerStarted","Data":"1e696ef7e374e82f10f3836c292176c3d08fcb41aaebfbb8c43cc16eb63f0e72"} Sep 10 16:00:42.863022 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:42.862987 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" event={"ID":"d7009d60-9919-4e28-a5e9-297b09620e31","Type":"ContainerStarted","Data":"cd9a64c3da56b520ed3b4058c8a5ec37280b0b102209946119765745a7b0c2c3"} Sep 10 16:00:43.867648 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:43.867615 2576 generic.go:356] "Generic (PLEG): container finished" podID="d7009d60-9919-4e28-a5e9-297b09620e31" containerID="cd9a64c3da56b520ed3b4058c8a5ec37280b0b102209946119765745a7b0c2c3" exitCode=0 Sep 10 16:00:43.868019 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:43.867656 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" event={"ID":"d7009d60-9919-4e28-a5e9-297b09620e31","Type":"ContainerDied","Data":"cd9a64c3da56b520ed3b4058c8a5ec37280b0b102209946119765745a7b0c2c3"} Sep 10 16:00:44.874245 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:44.874208 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" event={"ID":"d7009d60-9919-4e28-a5e9-297b09620e31","Type":"ContainerStarted","Data":"4ee153f6b42f525d3fd3024d97ebf0ac206f03be9746113f8fb57696a831658e"} Sep 10 16:00:44.874652 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:44.874360 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:44.896488 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:44.896431 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" podStartSLOduration=2.532683151 podStartE2EDuration="6.896410793s" totalImagesPullingTime="3.337798605s" totalInitContainerRuntime="1.025929037s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:00:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:00:44.895423858 +0000 UTC m=+287.671482326" watchObservedRunningTime="2026-09-10 16:00:44.896410793 +0000 UTC m=+287.672469225" Sep 10 16:00:55.880307 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:55.880279 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5995b6f449-65b7p" Sep 10 16:00:57.574545 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:57.574505 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 16:00:57.578503 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:57.578480 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 16:00:57.582146 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:00:57.582106 2576 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 16:01:15.721182 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.721148 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994"] Sep 10 16:01:15.724009 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.723991 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.726970 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.726840 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 16:01:15.726970 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.726852 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 16:01:15.726970 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.726936 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 16:01:15.727267 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.727211 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 16:01:15.727267 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.727225 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vghnx\"" Sep 10 16:01:15.727267 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.727255 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 16:01:15.727418 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.727251 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 16:01:15.748854 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.748826 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994"] Sep 10 16:01:15.832341 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.832306 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.832504 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.832355 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44bc083-f4b0-4c71-bb3a-fee853b35070-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.832504 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.832378 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtznp\" (UniqueName: \"kubernetes.io/projected/c44bc083-f4b0-4c71-bb3a-fee853b35070-kube-api-access-jtznp\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.832504 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.832413 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c44bc083-f4b0-4c71-bb3a-fee853b35070-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.832504 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.832434 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.832700 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.832517 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.832700 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.832562 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.933668 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.933634 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.933822 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.933678 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.933822 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.933725 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44bc083-f4b0-4c71-bb3a-fee853b35070-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.933822 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.933745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtznp\" (UniqueName: \"kubernetes.io/projected/c44bc083-f4b0-4c71-bb3a-fee853b35070-kube-api-access-jtznp\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.933822 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.933762 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c44bc083-f4b0-4c71-bb3a-fee853b35070-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.933822 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.933787 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.934064 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.933836 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.934321 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.934292 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.935888 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.935865 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44bc083-f4b0-4c71-bb3a-fee853b35070-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.936124 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.936104 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.936452 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.936435 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.936543 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.936512 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c44bc083-f4b0-4c71-bb3a-fee853b35070-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.951759 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.951737 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44bc083-f4b0-4c71-bb3a-fee853b35070-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:15.951913 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:15.951892 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtznp\" (UniqueName: \"kubernetes.io/projected/c44bc083-f4b0-4c71-bb3a-fee853b35070-kube-api-access-jtznp\") pod \"istiod-openshift-gateway-5677ccb49b-pc994\" (UID: \"c44bc083-f4b0-4c71-bb3a-fee853b35070\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:16.034968 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:16.034912 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:16.311487 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:16.311462 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994"] Sep 10 16:01:16.313851 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:01:16.313823 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc44bc083_f4b0_4c71_bb3a_fee853b35070.slice/crio-f1da6a61855fc62a6959b17467c9e4db23f3b7a133a0fb535973547a41981b6b WatchSource:0}: Error finding container f1da6a61855fc62a6959b17467c9e4db23f3b7a133a0fb535973547a41981b6b: Status 404 returned error can't find the container with id f1da6a61855fc62a6959b17467c9e4db23f3b7a133a0fb535973547a41981b6b Sep 10 16:01:16.315693 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:16.315680 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 16:01:16.990284 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:16.990252 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" event={"ID":"c44bc083-f4b0-4c71-bb3a-fee853b35070","Type":"ContainerStarted","Data":"f1da6a61855fc62a6959b17467c9e4db23f3b7a133a0fb535973547a41981b6b"} Sep 10 16:01:18.926231 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:18.926185 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 16:01:18.926496 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:18.926262 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 10 16:01:19.001297 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.001267 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" event={"ID":"c44bc083-f4b0-4c71-bb3a-fee853b35070","Type":"ContainerStarted","Data":"ae04139ef6fb08f1fefffdbc5150408357bb182c15ce18508ad6cf716328a64c"} Sep 10 16:01:19.001468 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.001454 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:19.002965 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.002941 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-pc994 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.18:8080/ready\": dial tcp 10.134.0.18:8080: connect: connection refused" start-of-body= Sep 10 16:01:19.003093 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.002981 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" podUID="c44bc083-f4b0-4c71-bb3a-fee853b35070" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.18:8080/ready\": dial tcp 10.134.0.18:8080: connect: connection refused" Sep 10 16:01:19.043404 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.043356 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" podStartSLOduration=1.433236075 podStartE2EDuration="4.043339485s" totalImagesPullingTime="2.61010341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:01:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:01:19.041058324 +0000 UTC m=+321.817116757" watchObservedRunningTime="2026-09-10 16:01:19.043339485 +0000 UTC m=+321.819397916" Sep 10 16:01:19.577214 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.577179 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj"] Sep 10 16:01:19.579815 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.579798 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.582749 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.582728 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 16:01:19.582868 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.582773 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 16:01:19.583997 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.583975 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 16:01:19.584093 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.584010 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 16:01:19.584155 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.584124 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xn6tz\"" Sep 10 16:01:19.584262 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.584246 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 16:01:19.589903 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.589883 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj"] Sep 10 16:01:19.660520 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.660497 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6jx8\" (UniqueName: \"kubernetes.io/projected/e1599453-cff7-4565-839e-af2e22c92443-kube-api-access-j6jx8\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.660614 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.660524 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1599453-cff7-4565-839e-af2e22c92443-manager-config\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.660614 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.660541 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1599453-cff7-4565-839e-af2e22c92443-cert\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.660614 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.660559 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1599453-cff7-4565-839e-af2e22c92443-metrics-cert\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.761275 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.761244 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1599453-cff7-4565-839e-af2e22c92443-metrics-cert\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.761397 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.761351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6jx8\" (UniqueName: \"kubernetes.io/projected/e1599453-cff7-4565-839e-af2e22c92443-kube-api-access-j6jx8\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.761397 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.761378 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1599453-cff7-4565-839e-af2e22c92443-manager-config\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.761495 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.761401 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1599453-cff7-4565-839e-af2e22c92443-cert\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.762129 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.762104 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1599453-cff7-4565-839e-af2e22c92443-manager-config\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.763694 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.763669 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1599453-cff7-4565-839e-af2e22c92443-cert\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.763839 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.763821 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1599453-cff7-4565-839e-af2e22c92443-metrics-cert\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.773117 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.773094 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6jx8\" (UniqueName: \"kubernetes.io/projected/e1599453-cff7-4565-839e-af2e22c92443-kube-api-access-j6jx8\") pod \"lws-controller-manager-5b9559cd76-gvnlj\" (UID: \"e1599453-cff7-4565-839e-af2e22c92443\") " pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:19.889468 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:19.889402 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:20.006123 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:20.006094 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-pc994 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 16:01:20.006437 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:20.006163 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" podUID="c44bc083-f4b0-4c71-bb3a-fee853b35070" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 16:01:20.166225 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:20.165924 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj"] Sep 10 16:01:20.168494 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:01:20.168463 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1599453_cff7_4565_839e_af2e22c92443.slice/crio-aab85044f3a9b295b5410e9c015ed1e2ff0d3c7cfb317050f9ee1787ce15e06f WatchSource:0}: Error finding container aab85044f3a9b295b5410e9c015ed1e2ff0d3c7cfb317050f9ee1787ce15e06f: Status 404 returned error can't find the container with id aab85044f3a9b295b5410e9c015ed1e2ff0d3c7cfb317050f9ee1787ce15e06f Sep 10 16:01:21.011749 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:21.011665 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" event={"ID":"e1599453-cff7-4565-839e-af2e22c92443","Type":"ContainerStarted","Data":"aab85044f3a9b295b5410e9c015ed1e2ff0d3c7cfb317050f9ee1787ce15e06f"} Sep 10 16:01:23.006520 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:23.006490 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pc994" Sep 10 16:01:23.021016 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:23.020991 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" event={"ID":"e1599453-cff7-4565-839e-af2e22c92443","Type":"ContainerStarted","Data":"a4b37f5f457b8d68a5824f32212e5a5b087cad47813b564ae0aeb6a4e83e5179"} Sep 10 16:01:23.021142 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:23.021122 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:01:23.044031 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:23.043979 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" podStartSLOduration=1.717151092 podStartE2EDuration="4.043963522s" totalImagesPullingTime="2.32681243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:01:23.042220519 +0000 UTC m=+325.818278951" watchObservedRunningTime="2026-09-10 16:01:23.043963522 +0000 UTC m=+325.820021954" Sep 10 16:01:34.026697 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:01:34.026664 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b9559cd76-gvnlj" Sep 10 16:02:29.404275 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.404199 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86c9fc6d64-279l7"] Sep 10 16:02:29.407153 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.407133 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.410347 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.410328 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jkblw\"" Sep 10 16:02:29.411587 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.411563 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 16:02:29.411587 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.411584 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 16:02:29.411835 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.411604 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 16:02:29.411835 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.411612 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 16:02:29.411835 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.411678 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 16:02:29.412770 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.412752 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 16:02:29.412875 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.412787 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 16:02:29.415884 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.415867 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 16:02:29.418092 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.418072 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c9fc6d64-279l7"] Sep 10 16:02:29.552181 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.552149 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppfz\" (UniqueName: \"kubernetes.io/projected/dbb41229-ec65-4584-b6a0-a5f55d162b72-kube-api-access-wppfz\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.552181 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.552189 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-oauth-serving-cert\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.552356 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.552260 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-trusted-ca-bundle\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.552356 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.552304 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbb41229-ec65-4584-b6a0-a5f55d162b72-console-serving-cert\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.552356 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.552321 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-console-config\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.552451 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.552364 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbb41229-ec65-4584-b6a0-a5f55d162b72-console-oauth-config\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.552451 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.552383 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-service-ca\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.652882 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.652848 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppfz\" (UniqueName: \"kubernetes.io/projected/dbb41229-ec65-4584-b6a0-a5f55d162b72-kube-api-access-wppfz\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.652882 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.652886 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-oauth-serving-cert\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.653054 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.652925 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-trusted-ca-bundle\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.653054 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.652952 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbb41229-ec65-4584-b6a0-a5f55d162b72-console-serving-cert\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.653054 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.652968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-console-config\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.653054 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.653004 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbb41229-ec65-4584-b6a0-a5f55d162b72-console-oauth-config\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.653054 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.653027 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-service-ca\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.653819 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.653797 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-console-config\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.653973 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.653802 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-oauth-serving-cert\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.653973 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.653892 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-service-ca\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.654096 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.653899 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb41229-ec65-4584-b6a0-a5f55d162b72-trusted-ca-bundle\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.655390 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.655344 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbb41229-ec65-4584-b6a0-a5f55d162b72-console-oauth-config\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.655484 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.655458 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbb41229-ec65-4584-b6a0-a5f55d162b72-console-serving-cert\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.661300 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.661282 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppfz\" (UniqueName: \"kubernetes.io/projected/dbb41229-ec65-4584-b6a0-a5f55d162b72-kube-api-access-wppfz\") pod \"console-86c9fc6d64-279l7\" (UID: \"dbb41229-ec65-4584-b6a0-a5f55d162b72\") " pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.716132 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.716111 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:29.997134 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:29.997095 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c9fc6d64-279l7"] Sep 10 16:02:30.000161 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:02:30.000133 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb41229_ec65_4584_b6a0_a5f55d162b72.slice/crio-ae6b14a5bad229c3fb21a83f90c746be96b583dbeca421272816fa2ea0f4de0b WatchSource:0}: Error finding container ae6b14a5bad229c3fb21a83f90c746be96b583dbeca421272816fa2ea0f4de0b: Status 404 returned error can't find the container with id ae6b14a5bad229c3fb21a83f90c746be96b583dbeca421272816fa2ea0f4de0b Sep 10 16:02:30.268884 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:30.268797 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9fc6d64-279l7" event={"ID":"dbb41229-ec65-4584-b6a0-a5f55d162b72","Type":"ContainerStarted","Data":"35ee110625ef7a9c1226be1ba56ffb5680d59813a5180733845f8efde6b6f9f8"} Sep 10 16:02:30.268884 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:30.268851 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9fc6d64-279l7" event={"ID":"dbb41229-ec65-4584-b6a0-a5f55d162b72","Type":"ContainerStarted","Data":"ae6b14a5bad229c3fb21a83f90c746be96b583dbeca421272816fa2ea0f4de0b"} Sep 10 16:02:30.289716 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:30.289667 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86c9fc6d64-279l7" podStartSLOduration=1.289645247 podStartE2EDuration="1.289645247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:02:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:30.289154476 +0000 UTC m=+393.065212905" watchObservedRunningTime="2026-09-10 16:02:30.289645247 +0000 UTC m=+393.065703680" Sep 10 16:02:39.716543 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:39.716513 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:39.716965 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:39.716556 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:39.721191 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:39.721168 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:02:40.309560 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:02:40.309527 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86c9fc6d64-279l7" Sep 10 16:03:57.039242 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.039209 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2"] Sep 10 16:03:57.042352 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.042335 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:03:57.045733 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.045699 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 16:03:57.045849 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.045804 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 10 16:03:57.046101 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.046085 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zp7sb\"" Sep 10 16:03:57.047069 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.047054 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 16:03:57.056810 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.056791 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2"] Sep 10 16:03:57.171400 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.171366 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwwkg\" (UniqueName: \"kubernetes.io/projected/ad5b93ee-241e-43d7-900c-65069dc6181b-kube-api-access-hwwkg\") pod \"postgres-69fb4594cf-zg5j2\" (UID: \"ad5b93ee-241e-43d7-900c-65069dc6181b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:03:57.171563 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.171423 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ad5b93ee-241e-43d7-900c-65069dc6181b-data\") pod \"postgres-69fb4594cf-zg5j2\" (UID: \"ad5b93ee-241e-43d7-900c-65069dc6181b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:03:57.272637 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.272611 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ad5b93ee-241e-43d7-900c-65069dc6181b-data\") pod \"postgres-69fb4594cf-zg5j2\" (UID: \"ad5b93ee-241e-43d7-900c-65069dc6181b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:03:57.272776 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.272662 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwwkg\" (UniqueName: \"kubernetes.io/projected/ad5b93ee-241e-43d7-900c-65069dc6181b-kube-api-access-hwwkg\") pod \"postgres-69fb4594cf-zg5j2\" (UID: \"ad5b93ee-241e-43d7-900c-65069dc6181b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:03:57.272991 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.272974 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ad5b93ee-241e-43d7-900c-65069dc6181b-data\") pod \"postgres-69fb4594cf-zg5j2\" (UID: \"ad5b93ee-241e-43d7-900c-65069dc6181b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:03:57.282632 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.282614 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwwkg\" (UniqueName: \"kubernetes.io/projected/ad5b93ee-241e-43d7-900c-65069dc6181b-kube-api-access-hwwkg\") pod \"postgres-69fb4594cf-zg5j2\" (UID: \"ad5b93ee-241e-43d7-900c-65069dc6181b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:03:57.352256 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.352198 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:03:57.644578 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:57.644507 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2"] Sep 10 16:03:57.647288 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:03:57.647259 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad5b93ee_241e_43d7_900c_65069dc6181b.slice/crio-f6cf9ff5359f8a78741b99888f3a6fe6f904fc2602212cbf6b37580e42e0ddce WatchSource:0}: Error finding container f6cf9ff5359f8a78741b99888f3a6fe6f904fc2602212cbf6b37580e42e0ddce: Status 404 returned error can't find the container with id f6cf9ff5359f8a78741b99888f3a6fe6f904fc2602212cbf6b37580e42e0ddce Sep 10 16:03:58.581790 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:03:58.581754 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" event={"ID":"ad5b93ee-241e-43d7-900c-65069dc6181b","Type":"ContainerStarted","Data":"f6cf9ff5359f8a78741b99888f3a6fe6f904fc2602212cbf6b37580e42e0ddce"} Sep 10 16:04:04.033462 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:04.033437 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 10 16:04:04.608685 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:04.608647 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" event={"ID":"ad5b93ee-241e-43d7-900c-65069dc6181b","Type":"ContainerStarted","Data":"28e8dedf2f076b190ce34baebfaccf7d5c5bf6d463adf106e5a5d898f534bbe8"} Sep 10 16:04:04.608850 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:04.608766 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:04:04.625432 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:04.625371 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" podStartSLOduration=1.243512074 podStartE2EDuration="7.625357886s" totalImagesPullingTime="6.381845812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:04:04.623444424 +0000 UTC m=+487.399502855" watchObservedRunningTime="2026-09-10 16:04:04.625357886 +0000 UTC m=+487.401416363" Sep 10 16:04:10.640317 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:10.640289 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zg5j2" Sep 10 16:04:18.502615 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.502581 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64477c8ccd-gxlzs"] Sep 10 16:04:18.521067 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.521044 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.524188 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.524152 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 16:04:18.524376 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.524359 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 16:04:18.524473 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.524366 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 16:04:18.524473 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.524446 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nhc5n\"" Sep 10 16:04:18.533886 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.533854 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64477c8ccd-gxlzs"] Sep 10 16:04:18.535355 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.535329 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-webhook-cert\") pod \"maas-controller-64477c8ccd-gxlzs\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.535479 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.535368 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-metrics-certs\") pod \"maas-controller-64477c8ccd-gxlzs\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.535479 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.535406 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz7n9\" (UniqueName: \"kubernetes.io/projected/5dfb0b92-8094-4461-8208-d354f3774480-kube-api-access-wz7n9\") pod \"maas-controller-64477c8ccd-gxlzs\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.636345 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.636315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz7n9\" (UniqueName: \"kubernetes.io/projected/5dfb0b92-8094-4461-8208-d354f3774480-kube-api-access-wz7n9\") pod \"maas-controller-64477c8ccd-gxlzs\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.636486 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.636392 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-webhook-cert\") pod \"maas-controller-64477c8ccd-gxlzs\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.636486 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.636418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-metrics-certs\") pod \"maas-controller-64477c8ccd-gxlzs\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.638876 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.638853 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-webhook-cert\") pod \"maas-controller-64477c8ccd-gxlzs\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.638979 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.638902 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-metrics-certs\") pod \"maas-controller-64477c8ccd-gxlzs\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.644669 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.644647 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz7n9\" (UniqueName: \"kubernetes.io/projected/5dfb0b92-8094-4461-8208-d354f3774480-kube-api-access-wz7n9\") pod \"maas-controller-64477c8ccd-gxlzs\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.658749 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.658728 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64fc66b857-7plgg"] Sep 10 16:04:18.660791 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.660777 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.673072 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.673049 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fc66b857-7plgg"] Sep 10 16:04:18.736810 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.736780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm8ps\" (UniqueName: \"kubernetes.io/projected/26ad43db-a63a-49aa-9e01-3aa9f14dd644-kube-api-access-fm8ps\") pod \"maas-controller-64fc66b857-7plgg\" (UID: \"26ad43db-a63a-49aa-9e01-3aa9f14dd644\") " pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.736925 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.736850 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26ad43db-a63a-49aa-9e01-3aa9f14dd644-metrics-certs\") pod \"maas-controller-64fc66b857-7plgg\" (UID: \"26ad43db-a63a-49aa-9e01-3aa9f14dd644\") " pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.736925 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.736890 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26ad43db-a63a-49aa-9e01-3aa9f14dd644-webhook-cert\") pod \"maas-controller-64fc66b857-7plgg\" (UID: \"26ad43db-a63a-49aa-9e01-3aa9f14dd644\") " pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.832308 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.832246 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:18.838168 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.838145 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm8ps\" (UniqueName: \"kubernetes.io/projected/26ad43db-a63a-49aa-9e01-3aa9f14dd644-kube-api-access-fm8ps\") pod \"maas-controller-64fc66b857-7plgg\" (UID: \"26ad43db-a63a-49aa-9e01-3aa9f14dd644\") " pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.838271 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.838184 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26ad43db-a63a-49aa-9e01-3aa9f14dd644-metrics-certs\") pod \"maas-controller-64fc66b857-7plgg\" (UID: \"26ad43db-a63a-49aa-9e01-3aa9f14dd644\") " pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.838271 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.838209 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26ad43db-a63a-49aa-9e01-3aa9f14dd644-webhook-cert\") pod \"maas-controller-64fc66b857-7plgg\" (UID: \"26ad43db-a63a-49aa-9e01-3aa9f14dd644\") " pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.840560 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.840531 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26ad43db-a63a-49aa-9e01-3aa9f14dd644-metrics-certs\") pod \"maas-controller-64fc66b857-7plgg\" (UID: \"26ad43db-a63a-49aa-9e01-3aa9f14dd644\") " pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.840560 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.840550 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26ad43db-a63a-49aa-9e01-3aa9f14dd644-webhook-cert\") pod \"maas-controller-64fc66b857-7plgg\" (UID: \"26ad43db-a63a-49aa-9e01-3aa9f14dd644\") " pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.846201 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.846178 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm8ps\" (UniqueName: \"kubernetes.io/projected/26ad43db-a63a-49aa-9e01-3aa9f14dd644-kube-api-access-fm8ps\") pod \"maas-controller-64fc66b857-7plgg\" (UID: \"26ad43db-a63a-49aa-9e01-3aa9f14dd644\") " pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:18.970913 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:18.970887 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:19.098574 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:19.098553 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64477c8ccd-gxlzs"] Sep 10 16:04:19.100666 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:04:19.100643 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfb0b92_8094_4461_8208_d354f3774480.slice/crio-8cf7fc86b1e7173f92f4cc442b3ab79152f59b71379fb36b8ea4689c726fe688 WatchSource:0}: Error finding container 8cf7fc86b1e7173f92f4cc442b3ab79152f59b71379fb36b8ea4689c726fe688: Status 404 returned error can't find the container with id 8cf7fc86b1e7173f92f4cc442b3ab79152f59b71379fb36b8ea4689c726fe688 Sep 10 16:04:19.242220 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:19.242142 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fc66b857-7plgg"] Sep 10 16:04:19.244781 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:04:19.244754 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ad43db_a63a_49aa_9e01_3aa9f14dd644.slice/crio-4a87a7bcb66ccbeb460ed1aa716e5d3e009bc18195d7c85ee9241ca49f599431 WatchSource:0}: Error finding container 4a87a7bcb66ccbeb460ed1aa716e5d3e009bc18195d7c85ee9241ca49f599431: Status 404 returned error can't find the container with id 4a87a7bcb66ccbeb460ed1aa716e5d3e009bc18195d7c85ee9241ca49f599431 Sep 10 16:04:19.673111 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:19.673071 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" event={"ID":"5dfb0b92-8094-4461-8208-d354f3774480","Type":"ContainerStarted","Data":"8cf7fc86b1e7173f92f4cc442b3ab79152f59b71379fb36b8ea4689c726fe688"} Sep 10 16:04:19.674197 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:19.674174 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fc66b857-7plgg" event={"ID":"26ad43db-a63a-49aa-9e01-3aa9f14dd644","Type":"ContainerStarted","Data":"4a87a7bcb66ccbeb460ed1aa716e5d3e009bc18195d7c85ee9241ca49f599431"} Sep 10 16:04:22.688557 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:22.688522 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" event={"ID":"5dfb0b92-8094-4461-8208-d354f3774480","Type":"ContainerStarted","Data":"b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56"} Sep 10 16:04:22.688984 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:22.688664 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:22.690061 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:22.690036 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fc66b857-7plgg" event={"ID":"26ad43db-a63a-49aa-9e01-3aa9f14dd644","Type":"ContainerStarted","Data":"3984134893ade8694a28d2f21859c3382f501d392b28fe41cafe9e30baf1db7e"} Sep 10 16:04:22.690139 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:22.690087 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:22.708722 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:22.708665 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" podStartSLOduration=2.023847856 podStartE2EDuration="4.708651663s" totalImagesPullingTime="2.684803807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:04:22.707028276 +0000 UTC m=+505.483086709" watchObservedRunningTime="2026-09-10 16:04:22.708651663 +0000 UTC m=+505.484710094" Sep 10 16:04:22.727895 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:22.727853 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64fc66b857-7plgg" podStartSLOduration=2.195200421 podStartE2EDuration="4.727839527s" totalImagesPullingTime="2.532639106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:04:22.726392381 +0000 UTC m=+505.502450839" watchObservedRunningTime="2026-09-10 16:04:22.727839527 +0000 UTC m=+505.503898003" Sep 10 16:04:33.698522 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:33.698492 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:33.699000 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:33.698562 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64fc66b857-7plgg" Sep 10 16:04:33.766820 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:33.766792 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64477c8ccd-gxlzs"] Sep 10 16:04:33.767051 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:33.766997 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" podUID="5dfb0b92-8094-4461-8208-d354f3774480" containerName="manager" containerID="cri-o://b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56" gracePeriod=30 Sep 10 16:04:34.006729 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.006693 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:34.065310 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.065281 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-metrics-certs\") pod \"5dfb0b92-8094-4461-8208-d354f3774480\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " Sep 10 16:04:34.065472 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.065320 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz7n9\" (UniqueName: \"kubernetes.io/projected/5dfb0b92-8094-4461-8208-d354f3774480-kube-api-access-wz7n9\") pod \"5dfb0b92-8094-4461-8208-d354f3774480\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " Sep 10 16:04:34.065472 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.065340 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-webhook-cert\") pod \"5dfb0b92-8094-4461-8208-d354f3774480\" (UID: \"5dfb0b92-8094-4461-8208-d354f3774480\") " Sep 10 16:04:34.067501 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.067470 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5dfb0b92-8094-4461-8208-d354f3774480" (UID: "5dfb0b92-8094-4461-8208-d354f3774480"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 16:04:34.067613 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.067483 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5dfb0b92-8094-4461-8208-d354f3774480" (UID: "5dfb0b92-8094-4461-8208-d354f3774480"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 16:04:34.067613 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.067559 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfb0b92-8094-4461-8208-d354f3774480-kube-api-access-wz7n9" (OuterVolumeSpecName: "kube-api-access-wz7n9") pod "5dfb0b92-8094-4461-8208-d354f3774480" (UID: "5dfb0b92-8094-4461-8208-d354f3774480"). InnerVolumeSpecName "kube-api-access-wz7n9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 16:04:34.166696 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.166670 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-metrics-certs\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 16:04:34.166696 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.166692 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz7n9\" (UniqueName: \"kubernetes.io/projected/5dfb0b92-8094-4461-8208-d354f3774480-kube-api-access-wz7n9\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 16:04:34.166866 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.166703 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfb0b92-8094-4461-8208-d354f3774480-webhook-cert\") on node \"ip-10-0-129-74.ec2.internal\" DevicePath \"\"" Sep 10 16:04:34.738898 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.738867 2576 generic.go:356] "Generic (PLEG): container finished" podID="5dfb0b92-8094-4461-8208-d354f3774480" containerID="b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56" exitCode=0 Sep 10 16:04:34.739355 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.738918 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" Sep 10 16:04:34.739355 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.738953 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" event={"ID":"5dfb0b92-8094-4461-8208-d354f3774480","Type":"ContainerDied","Data":"b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56"} Sep 10 16:04:34.739355 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.739004 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64477c8ccd-gxlzs" event={"ID":"5dfb0b92-8094-4461-8208-d354f3774480","Type":"ContainerDied","Data":"8cf7fc86b1e7173f92f4cc442b3ab79152f59b71379fb36b8ea4689c726fe688"} Sep 10 16:04:34.739355 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.739026 2576 scope.go:117] "RemoveContainer" containerID="b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56" Sep 10 16:04:34.747567 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.747551 2576 scope.go:117] "RemoveContainer" containerID="b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56" Sep 10 16:04:34.747871 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:04:34.747854 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56\": container with ID starting with b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56 not found: ID does not exist" containerID="b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56" Sep 10 16:04:34.747930 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.747883 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56"} err="failed to get container status \"b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56\": rpc error: code = NotFound desc = could not find container \"b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56\": container with ID starting with b4ef160ab4ceaa739012cc0edfbe4be4e4d708d097a95783e4cd5a63c188cc56 not found: ID does not exist" Sep 10 16:04:34.759974 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.759952 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64477c8ccd-gxlzs"] Sep 10 16:04:34.762689 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:34.762670 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64477c8ccd-gxlzs"] Sep 10 16:04:35.702013 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:35.701979 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dfb0b92-8094-4461-8208-d354f3774480" path="/var/lib/kubelet/pods/5dfb0b92-8094-4461-8208-d354f3774480/volumes" Sep 10 16:04:56.102563 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.102491 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d"] Sep 10 16:04:56.103003 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.102795 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dfb0b92-8094-4461-8208-d354f3774480" containerName="manager" Sep 10 16:04:56.103003 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.102808 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dfb0b92-8094-4461-8208-d354f3774480" containerName="manager" Sep 10 16:04:56.103003 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.102874 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dfb0b92-8094-4461-8208-d354f3774480" containerName="manager" Sep 10 16:04:56.106971 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.106952 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.109918 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.109894 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 10 16:04:56.110056 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.110007 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5prn6\"" Sep 10 16:04:56.110588 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.110573 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 16:04:56.120992 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.120971 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d"] Sep 10 16:04:56.234236 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.234207 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac107297-8d32-4109-937f-a955ae3c76e7-metrics-certs\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.234371 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.234240 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac107297-8d32-4109-937f-a955ae3c76e7-openshift-service-ca\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.234371 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.234258 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac107297-8d32-4109-937f-a955ae3c76e7-maas-api-tls\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.234371 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.234321 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-569kw\" (UniqueName: \"kubernetes.io/projected/ac107297-8d32-4109-937f-a955ae3c76e7-kube-api-access-569kw\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.335207 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.335180 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac107297-8d32-4109-937f-a955ae3c76e7-openshift-service-ca\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.335366 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.335211 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac107297-8d32-4109-937f-a955ae3c76e7-maas-api-tls\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.335366 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.335242 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-569kw\" (UniqueName: \"kubernetes.io/projected/ac107297-8d32-4109-937f-a955ae3c76e7-kube-api-access-569kw\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.335366 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.335305 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac107297-8d32-4109-937f-a955ae3c76e7-metrics-certs\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.335366 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:04:56.335357 2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Sep 10 16:04:56.335590 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:04:56.335442 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac107297-8d32-4109-937f-a955ae3c76e7-maas-api-tls podName:ac107297-8d32-4109-937f-a955ae3c76e7 nodeName:}" failed. No retries permitted until 2026-09-10 16:04:56.835419501 +0000 UTC m=+539.611477923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac107297-8d32-4109-937f-a955ae3c76e7-maas-api-tls") pod "maas-api-ccd8f6445-cc97d" (UID: "ac107297-8d32-4109-937f-a955ae3c76e7") : secret "maas-api-serving-cert" not found Sep 10 16:04:56.335936 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.335914 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac107297-8d32-4109-937f-a955ae3c76e7-openshift-service-ca\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.337754 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.337733 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac107297-8d32-4109-937f-a955ae3c76e7-metrics-certs\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.344905 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.344887 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-569kw\" (UniqueName: \"kubernetes.io/projected/ac107297-8d32-4109-937f-a955ae3c76e7-kube-api-access-569kw\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.840551 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.840521 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac107297-8d32-4109-937f-a955ae3c76e7-maas-api-tls\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:56.842892 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:56.842869 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac107297-8d32-4109-937f-a955ae3c76e7-maas-api-tls\") pod \"maas-api-ccd8f6445-cc97d\" (UID: \"ac107297-8d32-4109-937f-a955ae3c76e7\") " pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:57.016788 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:57.016753 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:04:57.290230 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:57.290203 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d"] Sep 10 16:04:57.292813 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:04:57.292778 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac107297_8d32_4109_937f_a955ae3c76e7.slice/crio-6b5232c4dc8eb6e8fbf34398b66c21e1a5385f2537b584a16d90ab1517c6cea1 WatchSource:0}: Error finding container 6b5232c4dc8eb6e8fbf34398b66c21e1a5385f2537b584a16d90ab1517c6cea1: Status 404 returned error can't find the container with id 6b5232c4dc8eb6e8fbf34398b66c21e1a5385f2537b584a16d90ab1517c6cea1 Sep 10 16:04:57.828653 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:04:57.828615 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" event={"ID":"ac107297-8d32-4109-937f-a955ae3c76e7","Type":"ContainerStarted","Data":"6b5232c4dc8eb6e8fbf34398b66c21e1a5385f2537b584a16d90ab1517c6cea1"} Sep 10 16:05:01.846440 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:01.846398 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" event={"ID":"ac107297-8d32-4109-937f-a955ae3c76e7","Type":"ContainerStarted","Data":"29d5dd21e1a90d6340174a9319ce9b763d4c5dc580f695888fe529cf7b04acf9"} Sep 10 16:05:01.846844 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:01.846539 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:05:01.868051 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:01.868004 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" podStartSLOduration=1.8044553620000001 podStartE2EDuration="5.867991285s" totalImagesPullingTime="4.063535923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:05:01.865798182 +0000 UTC m=+544.641856617" watchObservedRunningTime="2026-09-10 16:05:01.867991285 +0000 UTC m=+544.644049718" Sep 10 16:05:07.855868 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:07.855835 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-ccd8f6445-cc97d" Sep 10 16:05:36.787415 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.787383 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6"] Sep 10 16:05:36.789758 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.789736 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.793561 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.793533 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 16:05:36.793665 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.793580 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 16:05:36.793665 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.793534 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2h8fg\"" Sep 10 16:05:36.793908 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.793674 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 10 16:05:36.802410 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.802391 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6"] Sep 10 16:05:36.844083 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.844055 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.844197 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.844089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.844197 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.844124 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.844277 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.844193 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.844313 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.844268 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0be7f24-9161-4462-bed5-71d7fa6cb66f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.844362 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.844316 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx6lw\" (UniqueName: \"kubernetes.io/projected/e0be7f24-9161-4462-bed5-71d7fa6cb66f-kube-api-access-rx6lw\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.944935 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.944904 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.945049 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.944941 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.945049 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.944973 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.945049 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.945005 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.945049 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.945025 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0be7f24-9161-4462-bed5-71d7fa6cb66f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.945049 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.945041 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx6lw\" (UniqueName: \"kubernetes.io/projected/e0be7f24-9161-4462-bed5-71d7fa6cb66f-kube-api-access-rx6lw\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.945398 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.945377 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.945476 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.945406 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.945516 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.945475 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.947243 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.947226 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0be7f24-9161-4462-bed5-71d7fa6cb66f-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.947656 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.947621 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0be7f24-9161-4462-bed5-71d7fa6cb66f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:36.954265 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:36.954243 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx6lw\" (UniqueName: \"kubernetes.io/projected/e0be7f24-9161-4462-bed5-71d7fa6cb66f-kube-api-access-rx6lw\") pod \"e2e-distinct-2-simulated-kserve-859856b874-jn2f6\" (UID: \"e0be7f24-9161-4462-bed5-71d7fa6cb66f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:37.100517 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:37.100460 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:37.375592 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:37.375519 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6"] Sep 10 16:05:37.378416 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:05:37.378388 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0be7f24_9161_4462_bed5_71d7fa6cb66f.slice/crio-84b6ef46148279e6238461f662d6b9bb9f1ea478da46d89ee0e0c7ddcd8538c6 WatchSource:0}: Error finding container 84b6ef46148279e6238461f662d6b9bb9f1ea478da46d89ee0e0c7ddcd8538c6: Status 404 returned error can't find the container with id 84b6ef46148279e6238461f662d6b9bb9f1ea478da46d89ee0e0c7ddcd8538c6 Sep 10 16:05:37.995347 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:37.995307 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" event={"ID":"e0be7f24-9161-4462-bed5-71d7fa6cb66f","Type":"ContainerStarted","Data":"84b6ef46148279e6238461f662d6b9bb9f1ea478da46d89ee0e0c7ddcd8538c6"} Sep 10 16:05:39.000360 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:39.000322 2576 generic.go:356] "Generic (PLEG): container finished" podID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" containerID="264f2a8dafa20a1509aa7133ddc79e35d910e4f3b5274d47a4bd3d23d4df6d7e" exitCode=0 Sep 10 16:05:39.000729 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:39.000382 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" event={"ID":"e0be7f24-9161-4462-bed5-71d7fa6cb66f","Type":"ContainerDied","Data":"264f2a8dafa20a1509aa7133ddc79e35d910e4f3b5274d47a4bd3d23d4df6d7e"} Sep 10 16:05:39.000729 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:39.000622 2576 scope.go:117] "RemoveContainer" containerID="264f2a8dafa20a1509aa7133ddc79e35d910e4f3b5274d47a4bd3d23d4df6d7e" Sep 10 16:05:40.006536 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:40.006504 2576 generic.go:356] "Generic (PLEG): container finished" podID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" containerID="a563ca111be469fb0f30ce6015f83a95440f2ebe1268c0bb5fe6dd54b3ee9fb8" exitCode=0 Sep 10 16:05:40.006996 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:40.006582 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" event={"ID":"e0be7f24-9161-4462-bed5-71d7fa6cb66f","Type":"ContainerDied","Data":"a563ca111be469fb0f30ce6015f83a95440f2ebe1268c0bb5fe6dd54b3ee9fb8"} Sep 10 16:05:40.006996 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:40.006639 2576 scope.go:117] "RemoveContainer" containerID="264f2a8dafa20a1509aa7133ddc79e35d910e4f3b5274d47a4bd3d23d4df6d7e" Sep 10 16:05:40.006996 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:40.006905 2576 scope.go:117] "RemoveContainer" containerID="a563ca111be469fb0f30ce6015f83a95440f2ebe1268c0bb5fe6dd54b3ee9fb8" Sep 10 16:05:40.007153 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:05:40.007074 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:05:41.011596 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:41.011559 2576 scope.go:117] "RemoveContainer" containerID="a563ca111be469fb0f30ce6015f83a95440f2ebe1268c0bb5fe6dd54b3ee9fb8" Sep 10 16:05:41.011987 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:05:41.011720 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:05:47.101523 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:47.101486 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:47.101934 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:47.101550 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:47.101934 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:47.101566 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:47.101934 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:47.101580 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:05:47.102034 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:47.101960 2576 scope.go:117] "RemoveContainer" containerID="a563ca111be469fb0f30ce6015f83a95440f2ebe1268c0bb5fe6dd54b3ee9fb8" Sep 10 16:05:47.102136 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:05:47.102115 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:05:48.039691 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:48.039663 2576 scope.go:117] "RemoveContainer" containerID="a563ca111be469fb0f30ce6015f83a95440f2ebe1268c0bb5fe6dd54b3ee9fb8" Sep 10 16:05:48.039871 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:05:48.039833 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:05:57.601794 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:57.601754 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 16:05:57.603640 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:05:57.603614 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 16:06:01.694190 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:01.694151 2576 scope.go:117] "RemoveContainer" containerID="a563ca111be469fb0f30ce6015f83a95440f2ebe1268c0bb5fe6dd54b3ee9fb8" Sep 10 16:06:02.101909 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:02.101881 2576 generic.go:356] "Generic (PLEG): container finished" podID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" containerID="e2f58788fd1a1b79d2efe5ecd82e84f4030a2148a9a110dce1a07f48578c4853" exitCode=0 Sep 10 16:06:02.102033 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:02.101956 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" event={"ID":"e0be7f24-9161-4462-bed5-71d7fa6cb66f","Type":"ContainerDied","Data":"e2f58788fd1a1b79d2efe5ecd82e84f4030a2148a9a110dce1a07f48578c4853"} Sep 10 16:06:02.102033 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:02.102012 2576 scope.go:117] "RemoveContainer" containerID="a563ca111be469fb0f30ce6015f83a95440f2ebe1268c0bb5fe6dd54b3ee9fb8" Sep 10 16:06:02.102290 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:02.102276 2576 scope.go:117] "RemoveContainer" containerID="e2f58788fd1a1b79d2efe5ecd82e84f4030a2148a9a110dce1a07f48578c4853" Sep 10 16:06:02.102476 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:02.102454 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:06:07.100690 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:07.100653 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:06:07.101200 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:07.100767 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:06:07.101200 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:07.100792 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:06:07.101200 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:07.100815 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:06:07.101200 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:07.101095 2576 scope.go:117] "RemoveContainer" containerID="e2f58788fd1a1b79d2efe5ecd82e84f4030a2148a9a110dce1a07f48578c4853" Sep 10 16:06:07.101341 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:07.101260 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:06:07.123197 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:07.123176 2576 scope.go:117] "RemoveContainer" containerID="e2f58788fd1a1b79d2efe5ecd82e84f4030a2148a9a110dce1a07f48578c4853" Sep 10 16:06:07.123336 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:07.123320 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:06:20.387072 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.386998 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw"] Sep 10 16:06:20.389092 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.389075 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.391864 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.391817 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 10 16:06:20.401615 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.401592 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw"] Sep 10 16:06:20.576462 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.576427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.576634 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.576477 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bf8ccf0-4022-4052-b794-b8f90690e6ec-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.576634 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.576532 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.576634 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.576558 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.576634 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.576605 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.576875 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.576637 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c29c\" (UniqueName: \"kubernetes.io/projected/1bf8ccf0-4022-4052-b794-b8f90690e6ec-kube-api-access-2c29c\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.677193 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.677127 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bf8ccf0-4022-4052-b794-b8f90690e6ec-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.677193 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.677180 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.677366 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.677203 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.677366 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.677219 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.677366 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.677235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c29c\" (UniqueName: \"kubernetes.io/projected/1bf8ccf0-4022-4052-b794-b8f90690e6ec-kube-api-access-2c29c\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.677366 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.677284 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.677576 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.677556 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.677627 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.677607 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.677703 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.677685 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.679325 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.679307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bf8ccf0-4022-4052-b794-b8f90690e6ec-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.679587 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.679567 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bf8ccf0-4022-4052-b794-b8f90690e6ec-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.684924 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.684907 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c29c\" (UniqueName: \"kubernetes.io/projected/1bf8ccf0-4022-4052-b794-b8f90690e6ec-kube-api-access-2c29c\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjfbw\" (UID: \"1bf8ccf0-4022-4052-b794-b8f90690e6ec\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.699670 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.699650 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:20.968253 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.968224 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw"] Sep 10 16:06:20.971182 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:06:20.971157 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf8ccf0_4022_4052_b794_b8f90690e6ec.slice/crio-39a1e9dee97e348eb36f585ee8fbf1f422dfb4552cefe0db1dd29e58301f397a WatchSource:0}: Error finding container 39a1e9dee97e348eb36f585ee8fbf1f422dfb4552cefe0db1dd29e58301f397a: Status 404 returned error can't find the container with id 39a1e9dee97e348eb36f585ee8fbf1f422dfb4552cefe0db1dd29e58301f397a Sep 10 16:06:20.972463 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:20.972446 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 16:06:21.175345 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:21.175320 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" event={"ID":"1bf8ccf0-4022-4052-b794-b8f90690e6ec","Type":"ContainerStarted","Data":"39a1e9dee97e348eb36f585ee8fbf1f422dfb4552cefe0db1dd29e58301f397a"} Sep 10 16:06:22.190942 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:22.190897 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" event={"ID":"1bf8ccf0-4022-4052-b794-b8f90690e6ec","Type":"ContainerStarted","Data":"9acc0e68ccf91eb114df23b750d8b5b18417d9d2e248a4310d49d2528872af48"} Sep 10 16:06:22.694335 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:22.694304 2576 scope.go:117] "RemoveContainer" containerID="e2f58788fd1a1b79d2efe5ecd82e84f4030a2148a9a110dce1a07f48578c4853" Sep 10 16:06:23.197377 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:23.197305 2576 generic.go:356] "Generic (PLEG): container finished" podID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" containerID="2471708ab1306b6c701fb00728285324b0806c495267ff779531f5daa86762b6" exitCode=0 Sep 10 16:06:23.197817 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:23.197378 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" event={"ID":"e0be7f24-9161-4462-bed5-71d7fa6cb66f","Type":"ContainerDied","Data":"2471708ab1306b6c701fb00728285324b0806c495267ff779531f5daa86762b6"} Sep 10 16:06:23.197817 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:23.197440 2576 scope.go:117] "RemoveContainer" containerID="e2f58788fd1a1b79d2efe5ecd82e84f4030a2148a9a110dce1a07f48578c4853" Sep 10 16:06:23.197817 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:23.197795 2576 scope.go:117] "RemoveContainer" containerID="2471708ab1306b6c701fb00728285324b0806c495267ff779531f5daa86762b6" Sep 10 16:06:23.198012 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:23.197986 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:06:23.215988 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:23.215946 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" podStartSLOduration=3.036048443 podStartE2EDuration="3.215934242s" totalImagesPullingTime="179.885799ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:06:22.214021884 +0000 UTC m=+624.990080321" watchObservedRunningTime="2026-09-10 16:06:23.215934242 +0000 UTC m=+625.991992752" Sep 10 16:06:27.101303 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:27.101261 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:06:27.101704 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:27.101326 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:06:27.101704 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:27.101342 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:06:27.101704 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:27.101356 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:06:27.101704 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:27.101680 2576 scope.go:117] "RemoveContainer" containerID="2471708ab1306b6c701fb00728285324b0806c495267ff779531f5daa86762b6" Sep 10 16:06:27.101884 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:27.101861 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:06:30.700606 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:30.700575 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:30.701065 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:30.700632 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:30.712506 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:30.712484 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:31.242090 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.242057 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjfbw" Sep 10 16:06:31.886170 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.886138 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts"] Sep 10 16:06:31.889690 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.889672 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:31.892588 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.892570 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 10 16:06:31.896978 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.896954 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts"] Sep 10 16:06:31.965768 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.965739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:31.965881 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.965777 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7jcd\" (UniqueName: \"kubernetes.io/projected/9c2b4687-95a7-492f-809a-e403177ed72f-kube-api-access-s7jcd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:31.965881 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.965861 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:31.965953 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.965898 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:31.965953 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.965926 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:31.966020 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:31.965966 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c2b4687-95a7-492f-809a-e403177ed72f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.066995 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.066970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.067124 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.067005 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.067124 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.067024 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.067124 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.067056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c2b4687-95a7-492f-809a-e403177ed72f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.067124 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.067075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.067124 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.067095 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7jcd\" (UniqueName: \"kubernetes.io/projected/9c2b4687-95a7-492f-809a-e403177ed72f-kube-api-access-s7jcd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.067429 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.067406 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.067497 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.067417 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.067497 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.067472 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.069214 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.069183 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c2b4687-95a7-492f-809a-e403177ed72f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.069488 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.069469 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c2b4687-95a7-492f-809a-e403177ed72f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.075738 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.075691 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7jcd\" (UniqueName: \"kubernetes.io/projected/9c2b4687-95a7-492f-809a-e403177ed72f-kube-api-access-s7jcd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts\" (UID: \"9c2b4687-95a7-492f-809a-e403177ed72f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.201680 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.201657 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:32.472341 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:32.472317 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts"] Sep 10 16:06:33.243771 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:33.243736 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" event={"ID":"9c2b4687-95a7-492f-809a-e403177ed72f","Type":"ContainerStarted","Data":"3c59dfb3c0e2b668a25b5026cc1e9c28fa20cd185228d920b8eb0fd52dc3dbc6"} Sep 10 16:06:33.244180 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:33.243784 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" event={"ID":"9c2b4687-95a7-492f-809a-e403177ed72f","Type":"ContainerStarted","Data":"ce641558f47af8d9708f406ac688ed6432e637792bf8519fca78b1c1285218b5"} Sep 10 16:06:33.261600 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:33.261551 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" podStartSLOduration=2.083784465 podStartE2EDuration="2.261537705s" totalImagesPullingTime="177.75324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:06:33.260373614 +0000 UTC m=+636.036432058" watchObservedRunningTime="2026-09-10 16:06:33.261537705 +0000 UTC m=+636.037596135" Sep 10 16:06:38.887528 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.887496 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595"] Sep 10 16:06:38.890152 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.890131 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:38.892688 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.892666 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 16:06:38.901916 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.901892 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595"] Sep 10 16:06:38.921575 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.921551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:38.921690 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.921582 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:38.921690 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.921607 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:38.921690 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.921627 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:38.921876 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.921761 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jht5m\" (UniqueName: \"kubernetes.io/projected/0441e0b3-c04f-48ae-8aa2-0ff763389fed-kube-api-access-jht5m\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:38.921876 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:38.921859 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0441e0b3-c04f-48ae-8aa2-0ff763389fed-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.022384 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.022362 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jht5m\" (UniqueName: \"kubernetes.io/projected/0441e0b3-c04f-48ae-8aa2-0ff763389fed-kube-api-access-jht5m\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.022479 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.022415 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0441e0b3-c04f-48ae-8aa2-0ff763389fed-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.022479 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.022444 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.022479 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.022460 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.022654 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.022481 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.022654 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.022501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.022932 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.022910 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.022932 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.022927 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.023050 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.022966 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.024750 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.024724 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0441e0b3-c04f-48ae-8aa2-0ff763389fed-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.024992 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.024978 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0441e0b3-c04f-48ae-8aa2-0ff763389fed-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.030961 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.030935 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jht5m\" (UniqueName: \"kubernetes.io/projected/0441e0b3-c04f-48ae-8aa2-0ff763389fed-kube-api-access-jht5m\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595\" (UID: \"0441e0b3-c04f-48ae-8aa2-0ff763389fed\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.200775 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.200746 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:39.468409 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:39.468384 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595"] Sep 10 16:06:39.470449 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:06:39.470421 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0441e0b3_c04f_48ae_8aa2_0ff763389fed.slice/crio-49229876009d84991a4fa8f86d92da65c041ca4ddd83302a4b3c878bec67f1e8 WatchSource:0}: Error finding container 49229876009d84991a4fa8f86d92da65c041ca4ddd83302a4b3c878bec67f1e8: Status 404 returned error can't find the container with id 49229876009d84991a4fa8f86d92da65c041ca4ddd83302a4b3c878bec67f1e8 Sep 10 16:06:40.281940 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:40.281906 2576 generic.go:356] "Generic (PLEG): container finished" podID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" containerID="2046fa2ce2bb735d02ba3efa1768a9336eeceb57c12f7cb0f405c18d992480af" exitCode=0 Sep 10 16:06:40.282288 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:40.281991 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" event={"ID":"0441e0b3-c04f-48ae-8aa2-0ff763389fed","Type":"ContainerDied","Data":"2046fa2ce2bb735d02ba3efa1768a9336eeceb57c12f7cb0f405c18d992480af"} Sep 10 16:06:40.282288 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:40.282042 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" event={"ID":"0441e0b3-c04f-48ae-8aa2-0ff763389fed","Type":"ContainerStarted","Data":"49229876009d84991a4fa8f86d92da65c041ca4ddd83302a4b3c878bec67f1e8"} Sep 10 16:06:40.282288 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:40.282162 2576 scope.go:117] "RemoveContainer" containerID="2046fa2ce2bb735d02ba3efa1768a9336eeceb57c12f7cb0f405c18d992480af" Sep 10 16:06:40.693975 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:40.693946 2576 scope.go:117] "RemoveContainer" containerID="2471708ab1306b6c701fb00728285324b0806c495267ff779531f5daa86762b6" Sep 10 16:06:40.694135 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:40.694116 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:06:41.288499 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:41.288465 2576 generic.go:356] "Generic (PLEG): container finished" podID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" containerID="d9b259c09a5eeede301e006d9ed48be03a8bd6f29121a7060df03f995cd922b4" exitCode=0 Sep 10 16:06:41.288925 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:41.288530 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" event={"ID":"0441e0b3-c04f-48ae-8aa2-0ff763389fed","Type":"ContainerDied","Data":"d9b259c09a5eeede301e006d9ed48be03a8bd6f29121a7060df03f995cd922b4"} Sep 10 16:06:41.288925 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:41.288583 2576 scope.go:117] "RemoveContainer" containerID="2046fa2ce2bb735d02ba3efa1768a9336eeceb57c12f7cb0f405c18d992480af" Sep 10 16:06:41.288925 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:41.288773 2576 scope.go:117] "RemoveContainer" containerID="d9b259c09a5eeede301e006d9ed48be03a8bd6f29121a7060df03f995cd922b4" Sep 10 16:06:41.289061 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:41.288962 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:06:42.202598 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:42.202560 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:42.202792 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:42.202624 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:42.214839 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:42.214819 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:42.295169 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:42.295146 2576 scope.go:117] "RemoveContainer" containerID="d9b259c09a5eeede301e006d9ed48be03a8bd6f29121a7060df03f995cd922b4" Sep 10 16:06:42.295566 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:42.295356 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:06:42.306155 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:42.306132 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts" Sep 10 16:06:49.201718 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:49.201686 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:49.202068 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:49.201749 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:49.202068 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:49.201980 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:49.202068 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:49.202027 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:06:49.202068 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:49.202054 2576 scope.go:117] "RemoveContainer" containerID="d9b259c09a5eeede301e006d9ed48be03a8bd6f29121a7060df03f995cd922b4" Sep 10 16:06:49.202223 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:49.202203 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:06:49.325115 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:49.325091 2576 scope.go:117] "RemoveContainer" containerID="d9b259c09a5eeede301e006d9ed48be03a8bd6f29121a7060df03f995cd922b4" Sep 10 16:06:49.325252 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:49.325234 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:06:53.694236 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:06:53.694207 2576 scope.go:117] "RemoveContainer" containerID="2471708ab1306b6c701fb00728285324b0806c495267ff779531f5daa86762b6" Sep 10 16:06:53.694571 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:06:53.694364 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:07:01.694263 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:01.694225 2576 scope.go:117] "RemoveContainer" containerID="d9b259c09a5eeede301e006d9ed48be03a8bd6f29121a7060df03f995cd922b4" Sep 10 16:07:02.381233 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:02.381153 2576 generic.go:356] "Generic (PLEG): container finished" podID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" containerID="ba35e8c99a937b9b85ef548ef3eda0bb117aa3b68a6641ee47a5dbb94b1b9f15" exitCode=0 Sep 10 16:07:02.381233 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:02.381197 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" event={"ID":"0441e0b3-c04f-48ae-8aa2-0ff763389fed","Type":"ContainerDied","Data":"ba35e8c99a937b9b85ef548ef3eda0bb117aa3b68a6641ee47a5dbb94b1b9f15"} Sep 10 16:07:02.381418 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:02.381251 2576 scope.go:117] "RemoveContainer" containerID="d9b259c09a5eeede301e006d9ed48be03a8bd6f29121a7060df03f995cd922b4" Sep 10 16:07:02.381580 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:02.381563 2576 scope.go:117] "RemoveContainer" containerID="ba35e8c99a937b9b85ef548ef3eda0bb117aa3b68a6641ee47a5dbb94b1b9f15" Sep 10 16:07:02.381786 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:02.381765 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:07:08.694539 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:08.694509 2576 scope.go:117] "RemoveContainer" containerID="2471708ab1306b6c701fb00728285324b0806c495267ff779531f5daa86762b6" Sep 10 16:07:09.201328 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:09.201261 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:07:09.201328 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:09.201314 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:07:09.201328 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:09.201330 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:07:09.201515 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:09.201343 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:07:09.201666 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:09.201653 2576 scope.go:117] "RemoveContainer" containerID="ba35e8c99a937b9b85ef548ef3eda0bb117aa3b68a6641ee47a5dbb94b1b9f15" Sep 10 16:07:09.201839 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:09.201821 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:07:09.412578 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:09.412545 2576 generic.go:356] "Generic (PLEG): container finished" podID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" exitCode=0 Sep 10 16:07:09.412770 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:09.412610 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" event={"ID":"e0be7f24-9161-4462-bed5-71d7fa6cb66f","Type":"ContainerDied","Data":"3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c"} Sep 10 16:07:09.412770 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:09.412666 2576 scope.go:117] "RemoveContainer" containerID="2471708ab1306b6c701fb00728285324b0806c495267ff779531f5daa86762b6" Sep 10 16:07:09.413047 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:09.413031 2576 scope.go:117] "RemoveContainer" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" Sep 10 16:07:09.413279 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:09.413252 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:07:17.101455 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:17.101418 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:07:17.101903 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:17.101478 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:07:17.101903 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:17.101496 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:07:17.101903 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:17.101508 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:07:17.101903 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:17.101815 2576 scope.go:117] "RemoveContainer" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" Sep 10 16:07:17.102039 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:17.101972 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:07:21.694238 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:21.694203 2576 scope.go:117] "RemoveContainer" containerID="ba35e8c99a937b9b85ef548ef3eda0bb117aa3b68a6641ee47a5dbb94b1b9f15" Sep 10 16:07:21.694598 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:21.694361 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:07:28.694164 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:28.694137 2576 scope.go:117] "RemoveContainer" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" Sep 10 16:07:28.694531 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:28.694300 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:07:34.694154 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:34.694119 2576 scope.go:117] "RemoveContainer" containerID="ba35e8c99a937b9b85ef548ef3eda0bb117aa3b68a6641ee47a5dbb94b1b9f15" Sep 10 16:07:35.531372 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:35.531331 2576 generic.go:356] "Generic (PLEG): container finished" podID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" containerID="47def10e46f86f379b7776fc8d628c3c70b7c94f87c4296e2596fc8e85bb4144" exitCode=0 Sep 10 16:07:35.531528 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:35.531379 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" event={"ID":"0441e0b3-c04f-48ae-8aa2-0ff763389fed","Type":"ContainerDied","Data":"47def10e46f86f379b7776fc8d628c3c70b7c94f87c4296e2596fc8e85bb4144"} Sep 10 16:07:35.531528 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:35.531451 2576 scope.go:117] "RemoveContainer" containerID="ba35e8c99a937b9b85ef548ef3eda0bb117aa3b68a6641ee47a5dbb94b1b9f15" Sep 10 16:07:35.531826 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:35.531796 2576 scope.go:117] "RemoveContainer" containerID="47def10e46f86f379b7776fc8d628c3c70b7c94f87c4296e2596fc8e85bb4144" Sep 10 16:07:35.531994 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:35.531965 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:07:39.201237 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:39.201193 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:07:39.201639 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:39.201273 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:07:39.201639 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:39.201297 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:07:39.201639 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:39.201319 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:07:39.201768 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:39.201674 2576 scope.go:117] "RemoveContainer" containerID="47def10e46f86f379b7776fc8d628c3c70b7c94f87c4296e2596fc8e85bb4144" Sep 10 16:07:39.201878 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:39.201859 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:07:42.694316 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:42.694286 2576 scope.go:117] "RemoveContainer" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" Sep 10 16:07:42.694738 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:42.694458 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:07:53.694691 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:53.694598 2576 scope.go:117] "RemoveContainer" containerID="47def10e46f86f379b7776fc8d628c3c70b7c94f87c4296e2596fc8e85bb4144" Sep 10 16:07:53.695236 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:53.694844 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:07:56.694823 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:07:56.694788 2576 scope.go:117] "RemoveContainer" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" Sep 10 16:07:56.695230 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:07:56.694986 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:08:05.694202 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:05.694169 2576 scope.go:117] "RemoveContainer" containerID="47def10e46f86f379b7776fc8d628c3c70b7c94f87c4296e2596fc8e85bb4144" Sep 10 16:08:05.694557 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:05.694341 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:08:07.700250 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:07.700215 2576 scope.go:117] "RemoveContainer" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" Sep 10 16:08:07.700727 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:07.700508 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:08:18.693833 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:18.693803 2576 scope.go:117] "RemoveContainer" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" Sep 10 16:08:18.694205 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:18.693964 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:08:20.694059 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:20.694032 2576 scope.go:117] "RemoveContainer" containerID="47def10e46f86f379b7776fc8d628c3c70b7c94f87c4296e2596fc8e85bb4144" Sep 10 16:08:21.727776 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:21.727742 2576 generic.go:356] "Generic (PLEG): container finished" podID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" exitCode=0 Sep 10 16:08:21.728106 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:21.727813 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" event={"ID":"0441e0b3-c04f-48ae-8aa2-0ff763389fed","Type":"ContainerDied","Data":"8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d"} Sep 10 16:08:21.728106 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:21.727871 2576 scope.go:117] "RemoveContainer" containerID="47def10e46f86f379b7776fc8d628c3c70b7c94f87c4296e2596fc8e85bb4144" Sep 10 16:08:21.728239 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:21.728228 2576 scope.go:117] "RemoveContainer" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" Sep 10 16:08:21.728417 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:21.728397 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:08:29.201342 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:29.201295 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:08:29.201850 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:29.201378 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:08:29.201850 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:29.201401 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:08:29.201850 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:29.201421 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:08:29.201850 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:29.201781 2576 scope.go:117] "RemoveContainer" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" Sep 10 16:08:29.201985 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:29.201945 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:08:29.694466 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:29.694433 2576 scope.go:117] "RemoveContainer" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" Sep 10 16:08:30.768347 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:30.768317 2576 generic.go:356] "Generic (PLEG): container finished" podID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" exitCode=0 Sep 10 16:08:30.768746 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:30.768371 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" event={"ID":"e0be7f24-9161-4462-bed5-71d7fa6cb66f","Type":"ContainerDied","Data":"1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46"} Sep 10 16:08:30.768746 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:30.768418 2576 scope.go:117] "RemoveContainer" containerID="3f3b007957982ff2ebe70d4f61145d1c1cf869a04447846d7a99fc98bb41918c" Sep 10 16:08:30.768888 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:30.768757 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:08:30.768968 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:30.768943 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:08:37.100724 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:37.100682 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:08:37.101169 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:37.100784 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:08:37.101169 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:37.100810 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:08:37.101169 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:37.100832 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" Sep 10 16:08:37.101358 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:37.101246 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:08:37.101516 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:37.101492 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:08:42.694254 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:42.694214 2576 scope.go:117] "RemoveContainer" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" Sep 10 16:08:42.694741 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:42.694450 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:08:50.694145 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:50.694116 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:08:50.694510 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:50.694289 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:08:56.693792 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:08:56.693763 2576 scope.go:117] "RemoveContainer" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" Sep 10 16:08:56.694149 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:08:56.693924 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:09:04.694174 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:04.694140 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:09:04.694614 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:04.694306 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:09:11.693896 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:11.693866 2576 scope.go:117] "RemoveContainer" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" Sep 10 16:09:11.694266 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:11.694048 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:09:17.695861 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:17.695820 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:09:17.696219 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:17.696013 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:09:26.694293 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:26.694203 2576 scope.go:117] "RemoveContainer" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" Sep 10 16:09:26.694756 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:26.694401 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:09:29.693655 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:29.693630 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:09:29.694019 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:29.693799 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:09:38.694288 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:38.694258 2576 scope.go:117] "RemoveContainer" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" Sep 10 16:09:38.694746 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:38.694432 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:09:40.694671 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:40.694644 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:09:40.695025 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:40.694842 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:09:49.694603 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:49.694568 2576 scope.go:117] "RemoveContainer" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" Sep 10 16:09:50.096391 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:50.096358 2576 generic.go:356] "Generic (PLEG): container finished" podID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" containerID="dc9a2c83fd88bb7d92e147fef835440490459929aad9a0cc3f396f5ee421a52d" exitCode=0 Sep 10 16:09:50.096551 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:50.096431 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" event={"ID":"0441e0b3-c04f-48ae-8aa2-0ff763389fed","Type":"ContainerDied","Data":"dc9a2c83fd88bb7d92e147fef835440490459929aad9a0cc3f396f5ee421a52d"} Sep 10 16:09:50.096551 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:50.096498 2576 scope.go:117] "RemoveContainer" containerID="8038ca0f1ebf89addf4fba73f113c10b1144c18c760c711ff216289de46cf45d" Sep 10 16:09:50.096921 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:50.096903 2576 scope.go:117] "RemoveContainer" containerID="dc9a2c83fd88bb7d92e147fef835440490459929aad9a0cc3f396f5ee421a52d" Sep 10 16:09:50.097103 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:50.097085 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:09:53.694437 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:53.694405 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:09:53.694920 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:53.694571 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:09:59.201696 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:59.201663 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:09:59.202131 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:59.201743 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:09:59.202131 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:59.201768 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:09:59.202131 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:59.201789 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" Sep 10 16:09:59.202131 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:09:59.202093 2576 scope.go:117] "RemoveContainer" containerID="dc9a2c83fd88bb7d92e147fef835440490459929aad9a0cc3f396f5ee421a52d" Sep 10 16:09:59.202340 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:09:59.202266 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:10:06.693787 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:06.693756 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:10:06.694147 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:10:06.693927 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:10:11.694273 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:11.694244 2576 scope.go:117] "RemoveContainer" containerID="dc9a2c83fd88bb7d92e147fef835440490459929aad9a0cc3f396f5ee421a52d" Sep 10 16:10:11.694703 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:10:11.694445 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:10:12.403983 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:12.403954 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_0441e0b3-c04f-48ae-8aa2-0ff763389fed/main/5.log" Sep 10 16:10:12.534142 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:12.534111 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_0441e0b3-c04f-48ae-8aa2-0ff763389fed/main/5.log" Sep 10 16:10:19.308154 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:19.308125 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64fc66b857-7plgg_26ad43db-a63a-49aa-9e01-3aa9f14dd644/manager/0.log" Sep 10 16:10:19.693652 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:19.693621 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:10:19.693872 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:10:19.693849 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:10:19.701050 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:19.701031 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5995b6f449-65b7p_d7009d60-9919-4e28-a5e9-297b09620e31/copy-manifests/0.log" Sep 10 16:10:19.724558 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:19.724535 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5995b6f449-65b7p_d7009d60-9919-4e28-a5e9-297b09620e31/manager/0.log" Sep 10 16:10:20.309454 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:20.309425 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-ccd8f6445-cc97d_ac107297-8d32-4109-937f-a955ae3c76e7/maas-api/0.log" Sep 10 16:10:20.426667 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:20.426641 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-zg5j2_ad5b93ee-241e-43d7-900c-65069dc6181b/postgres/0.log" Sep 10 16:10:22.388445 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:22.388411 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pc994_c44bc083-f4b0-4c71-bb3a-fee853b35070/discovery/0.log" Sep 10 16:10:22.693850 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:22.693821 2576 scope.go:117] "RemoveContainer" containerID="dc9a2c83fd88bb7d92e147fef835440490459929aad9a0cc3f396f5ee421a52d" Sep 10 16:10:22.694026 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:10:22.693969 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:10:23.438248 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:23.438218 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-859856b874-jn2f6_e0be7f24-9161-4462-bed5-71d7fa6cb66f/main/5.log" Sep 10 16:10:23.785080 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:23.785049 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vjfbw_1bf8ccf0-4022-4052-b794-b8f90690e6ec/main/0.log" Sep 10 16:10:23.900454 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:23.900422 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft69ts_9c2b4687-95a7-492f-809a-e403177ed72f/main/0.log" Sep 10 16:10:24.010107 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:24.010082 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_0441e0b3-c04f-48ae-8aa2-0ff763389fed/main/5.log" Sep 10 16:10:32.665407 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:32.665379 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q49dm_466e913c-b578-40aa-b099-669bdfe2ecd5/global-pull-secret-syncer/0.log" Sep 10 16:10:32.693801 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:32.693779 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:10:32.693957 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:10:32.693940 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:10:32.779353 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:32.779329 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w9mp9_954fee05-d00a-467e-b33b-3f16ded2d70a/konnectivity-agent/0.log" Sep 10 16:10:32.802627 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:32.802602 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-74.ec2.internal_e59b8d96466349d07517a47588dc5f77/haproxy/0.log" Sep 10 16:10:34.694552 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:34.694524 2576 scope.go:117] "RemoveContainer" containerID="dc9a2c83fd88bb7d92e147fef835440490459929aad9a0cc3f396f5ee421a52d" Sep 10 16:10:34.694936 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:10:34.694684 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:10:40.336829 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.336751 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b88jf_e1fc51db-e496-4b33-989b-e2b1e8c10890/node-exporter/0.log" Sep 10 16:10:40.361626 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.361599 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b88jf_e1fc51db-e496-4b33-989b-e2b1e8c10890/kube-rbac-proxy/0.log" Sep 10 16:10:40.392809 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.392785 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b88jf_e1fc51db-e496-4b33-989b-e2b1e8c10890/init-textfile/0.log" Sep 10 16:10:40.777872 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.777831 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40be2d3d-5c49-4ebf-bc91-65bffd1a898c/prometheus/0.log" Sep 10 16:10:40.808277 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.808256 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40be2d3d-5c49-4ebf-bc91-65bffd1a898c/config-reloader/0.log" Sep 10 16:10:40.853724 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.853691 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40be2d3d-5c49-4ebf-bc91-65bffd1a898c/thanos-sidecar/0.log" Sep 10 16:10:40.873569 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.873546 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40be2d3d-5c49-4ebf-bc91-65bffd1a898c/kube-rbac-proxy-web/0.log" Sep 10 16:10:40.902813 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.902791 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40be2d3d-5c49-4ebf-bc91-65bffd1a898c/kube-rbac-proxy/0.log" Sep 10 16:10:40.939642 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.939615 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40be2d3d-5c49-4ebf-bc91-65bffd1a898c/kube-rbac-proxy-thanos/0.log" Sep 10 16:10:40.971324 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:40.971302 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_40be2d3d-5c49-4ebf-bc91-65bffd1a898c/init-config-reloader/0.log" Sep 10 16:10:41.098050 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:41.097992 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-2wjvj_a6bde5d3-367d-449b-b159-e32ac7611052/prometheus-operator-admission-webhook/0.log" Sep 10 16:10:42.597331 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.597299 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc"] Sep 10 16:10:42.600667 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.600647 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.604214 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.604193 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2427l\"/\"kube-root-ca.crt\"" Sep 10 16:10:42.604326 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.604200 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2427l\"/\"openshift-service-ca.crt\"" Sep 10 16:10:42.605368 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.605350 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2427l\"/\"default-dockercfg-sq2jl\"" Sep 10 16:10:42.627091 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.626904 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-podres\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.627091 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.626975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftqwd\" (UniqueName: \"kubernetes.io/projected/bf221016-dcbd-4040-8974-c044f2969e78-kube-api-access-ftqwd\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.627091 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.627014 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-sys\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.627091 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.627055 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-lib-modules\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.627091 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.627081 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-proc\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.628457 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.628437 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc"] Sep 10 16:10:42.727830 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.727800 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-proc\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.727993 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.727858 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-podres\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.727993 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.727890 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftqwd\" (UniqueName: \"kubernetes.io/projected/bf221016-dcbd-4040-8974-c044f2969e78-kube-api-access-ftqwd\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.727993 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.727913 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-sys\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.727993 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.727919 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-proc\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.727993 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.727936 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-lib-modules\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.728242 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.727995 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-sys\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.728242 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.728052 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-lib-modules\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.728242 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.728056 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bf221016-dcbd-4040-8974-c044f2969e78-podres\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.739993 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.739971 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftqwd\" (UniqueName: \"kubernetes.io/projected/bf221016-dcbd-4040-8974-c044f2969e78-kube-api-access-ftqwd\") pod \"perf-node-gather-daemonset-bbbrc\" (UID: \"bf221016-dcbd-4040-8974-c044f2969e78\") " pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:42.910443 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:42.910369 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:43.214022 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:43.213978 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc"] Sep 10 16:10:43.215180 ip-10-0-129-74 kubenswrapper[2576]: W0910 16:10:43.215155 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbf221016_dcbd_4040_8974_c044f2969e78.slice/crio-5ac1acb3017d8a8e449b21f0d6bf1bbae1adfcdfaac7551b3ffad4f9fa6dda36 WatchSource:0}: Error finding container 5ac1acb3017d8a8e449b21f0d6bf1bbae1adfcdfaac7551b3ffad4f9fa6dda36: Status 404 returned error can't find the container with id 5ac1acb3017d8a8e449b21f0d6bf1bbae1adfcdfaac7551b3ffad4f9fa6dda36 Sep 10 16:10:43.320270 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:43.320241 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" event={"ID":"bf221016-dcbd-4040-8974-c044f2969e78","Type":"ContainerStarted","Data":"76811a0dfbceb398337e375e447eed4c5a91ae8f0ae29c1bc061abd5f779b8ba"} Sep 10 16:10:43.320357 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:43.320290 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" event={"ID":"bf221016-dcbd-4040-8974-c044f2969e78","Type":"ContainerStarted","Data":"5ac1acb3017d8a8e449b21f0d6bf1bbae1adfcdfaac7551b3ffad4f9fa6dda36"} Sep 10 16:10:43.320425 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:43.320398 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:43.347817 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:43.347776 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" podStartSLOduration=1.347760898 podStartE2EDuration="1.347760898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:10:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:10:43.3456375 +0000 UTC m=+886.121695932" watchObservedRunningTime="2026-09-10 16:10:43.347760898 +0000 UTC m=+886.123819369" Sep 10 16:10:44.058152 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:44.058122 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86c9fc6d64-279l7_dbb41229-ec65-4584-b6a0-a5f55d162b72/console/0.log" Sep 10 16:10:45.693891 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:45.693860 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:10:45.694228 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:10:45.694029 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:10:45.918274 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:45.918235 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tjh7r_f11d3e7f-f422-46de-aff9-fcd9448b183f/dns/0.log" Sep 10 16:10:45.941034 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:45.941012 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tjh7r_f11d3e7f-f422-46de-aff9-fcd9448b183f/kube-rbac-proxy/0.log" Sep 10 16:10:45.995480 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:45.995428 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-86g67_0a39261f-d1f2-438d-a6ef-b83066bbead7/dns-node-resolver/0.log" Sep 10 16:10:46.726145 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:46.726117 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sms7n_7438a602-18a4-48c3-aa99-7a3cdd98cae7/node-ca/0.log" Sep 10 16:10:47.695739 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:47.695694 2576 scope.go:117] "RemoveContainer" containerID="dc9a2c83fd88bb7d92e147fef835440490459929aad9a0cc3f396f5ee421a52d" Sep 10 16:10:47.695907 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:10:47.695872 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595_llm(0441e0b3-c04f-48ae-8aa2-0ff763389fed)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-pd595" podUID="0441e0b3-c04f-48ae-8aa2-0ff763389fed" Sep 10 16:10:47.807512 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:47.807483 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pc994_c44bc083-f4b0-4c71-bb3a-fee853b35070/discovery/0.log" Sep 10 16:10:48.673541 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:48.673515 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4m2cz_b703f5eb-7202-4225-8193-a0b36bc187d6/serve-healthcheck-canary/0.log" Sep 10 16:10:49.298441 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:49.298411 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-59zmf_35285e79-c766-484e-982b-85f498f2d02c/kube-rbac-proxy/0.log" Sep 10 16:10:49.323810 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:49.323786 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-59zmf_35285e79-c766-484e-982b-85f498f2d02c/exporter/0.log" Sep 10 16:10:49.333609 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:49.333590 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2427l/perf-node-gather-daemonset-bbbrc" Sep 10 16:10:49.347217 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:49.347199 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-59zmf_35285e79-c766-484e-982b-85f498f2d02c/extractor/0.log" Sep 10 16:10:51.962131 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:51.962097 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64fc66b857-7plgg_26ad43db-a63a-49aa-9e01-3aa9f14dd644/manager/0.log" Sep 10 16:10:52.178396 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:52.178325 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5995b6f449-65b7p_d7009d60-9919-4e28-a5e9-297b09620e31/manager/0.log" Sep 10 16:10:52.202183 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:52.202158 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5995b6f449-65b7p_d7009d60-9919-4e28-a5e9-297b09620e31/copy-manifests/0.log" Sep 10 16:10:53.743273 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:53.743240 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b9559cd76-gvnlj_e1599453-cff7-4565-839e-af2e22c92443/manager/0.log" Sep 10 16:10:53.901082 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:53.901054 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pc994_c44bc083-f4b0-4c71-bb3a-fee853b35070/discovery/0.log" Sep 10 16:10:55.450628 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:55.450598 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pc994_c44bc083-f4b0-4c71-bb3a-fee853b35070/discovery/0.log" Sep 10 16:10:56.694498 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:56.694464 2576 scope.go:117] "RemoveContainer" containerID="1284217c5968320a1ae4b3523191b39b539d736db0c3103bd35d0d28e6c2ee46" Sep 10 16:10:56.694914 ip-10-0-129-74 kubenswrapper[2576]: E0910 16:10:56.694625 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-jn2f6_llm(e0be7f24-9161-4462-bed5-71d7fa6cb66f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-jn2f6" podUID="e0be7f24-9161-4462-bed5-71d7fa6cb66f" Sep 10 16:10:57.627991 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:57.627959 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 16:10:57.628178 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:10:57.628051 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zwpx_5a42ae36-0842-4ae5-8d49-a00d603fdd5b/ovn-acl-logging/0.log" Sep 10 16:11:00.411855 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:11:00.411822 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4kfq7_b03fdcb9-b788-4551-8f73-b5cce9a1e52c/kube-multus/0.log" Sep 10 16:11:00.607921 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:11:00.607892 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nq9fq_e92b2fbf-b2f0-476b-842a-93d60ce94b8c/kube-multus-additional-cni-plugins/0.log" Sep 10 16:11:00.628329 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:11:00.628302 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nq9fq_e92b2fbf-b2f0-476b-842a-93d60ce94b8c/egress-router-binary-copy/0.log" Sep 10 16:11:00.648793 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:11:00.648765 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nq9fq_e92b2fbf-b2f0-476b-842a-93d60ce94b8c/cni-plugins/0.log" Sep 10 16:11:00.674455 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:11:00.674396 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nq9fq_e92b2fbf-b2f0-476b-842a-93d60ce94b8c/bond-cni-plugin/0.log" Sep 10 16:11:00.695263 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:11:00.695238 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nq9fq_e92b2fbf-b2f0-476b-842a-93d60ce94b8c/routeoverride-cni/0.log" Sep 10 16:11:00.716566 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:11:00.716543 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nq9fq_e92b2fbf-b2f0-476b-842a-93d60ce94b8c/whereabouts-cni-bincopy/0.log" Sep 10 16:11:00.737975 ip-10-0-129-74 kubenswrapper[2576]: I0910 16:11:00.737953 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nq9fq_e92b2fbf-b2f0-476b-842a-93d60ce94b8c/whereabouts-cni/0.log"