Sep 05 02:17:55.378898 ip-10-0-128-63 systemd[1]: Starting Kubernetes Kubelet... Sep 05 02:17:55.785233 ip-10-0-128-63 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 05 02:17:55.785233 ip-10-0-128-63 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 05 02:17:55.785233 ip-10-0-128-63 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 05 02:17:55.785233 ip-10-0-128-63 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 05 02:17:55.785233 ip-10-0-128-63 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 05 02:17:55.787677 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.787591 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 05 02:17:55.792523 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792508 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 02:17:55.792523 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792524 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792528 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792531 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792534 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792538 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792541 2567 feature_gate.go:328] unrecognized feature gate: Example Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792544 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792551 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792554 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792557 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792560 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792562 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792565 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792568 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792583 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792585 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792588 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792591 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792594 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792597 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 02:17:55.792607 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792600 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792602 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792605 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792607 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792610 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792613 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792615 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792619 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792621 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792624 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792626 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792629 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792631 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792634 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792636 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792639 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792641 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792645 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792647 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792650 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 02:17:55.793081 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792652 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792655 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792657 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792660 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792662 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792665 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792667 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792669 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792674 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792679 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792682 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792685 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792687 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792690 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792694 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792697 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792700 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792702 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792705 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792708 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 02:17:55.793615 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792710 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792714 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792717 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792721 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792724 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792727 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792730 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792732 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792735 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792738 2567 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792741 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792744 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792747 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792749 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792751 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792754 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792756 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792759 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792762 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 02:17:55.794112 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792764 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792766 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792769 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792771 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792774 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792776 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792781 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792784 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792787 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792790 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792793 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792796 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792798 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792801 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792804 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792806 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792809 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792812 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.792815 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 02:17:55.794614 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793498 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793505 2567 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793508 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793511 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793514 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793517 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793521 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793523 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793526 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793529 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793533 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793537 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793540 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793543 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793546 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793549 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793552 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793554 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793558 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793561 2567 feature_gate.go:328] unrecognized feature gate: Example Sep 05 02:17:55.795069 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793564 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793567 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793585 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793588 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793591 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793594 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793597 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793600 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793602 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793605 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793608 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793610 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793613 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793615 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793618 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793621 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793624 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793628 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793634 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793637 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 02:17:55.795540 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793640 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793643 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793646 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793648 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793651 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793654 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793656 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793660 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793663 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793665 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793668 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793671 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793674 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793676 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793679 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793682 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793686 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793689 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 02:17:55.796072 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793692 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793695 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793698 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793700 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793703 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793705 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793708 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793711 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793714 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793716 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793719 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793721 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793724 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793727 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793729 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793732 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793734 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793740 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793742 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793745 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 02:17:55.796501 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793747 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793750 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793752 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793755 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793758 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793760 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793763 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793766 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793768 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793771 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793773 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793776 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793778 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793781 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793783 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793786 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793788 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793792 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793795 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793797 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 02:17:55.796996 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.793800 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794194 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794198 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794201 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794203 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794207 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794210 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794212 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794215 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794219 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794222 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794224 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794227 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794229 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794232 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794235 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794238 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794240 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794243 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 02:17:55.797481 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794246 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794248 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794251 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794253 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794256 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794258 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794261 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794263 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794266 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794268 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794271 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794273 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794276 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794279 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794281 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794283 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794286 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794289 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794291 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794294 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 02:17:55.797973 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794296 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794301 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794305 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794308 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794311 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794314 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794317 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794319 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794322 2567 feature_gate.go:328] unrecognized feature gate: Example Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794326 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794329 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794332 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794334 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794344 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794348 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794350 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794353 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794356 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794358 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794361 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 02:17:55.798465 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794363 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794366 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794368 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794371 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794373 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794375 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794378 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794381 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794385 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794388 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794391 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794393 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794397 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794399 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794404 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794406 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794409 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794411 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794414 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794416 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 02:17:55.798999 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794418 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794422 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794424 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794427 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794430 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794432 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794434 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794437 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794439 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794442 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794444 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794446 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794449 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794451 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794454 2567 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794456 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794458 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794461 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794464 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794468 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 02:17:55.799472 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.794470 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794542 2567 flags.go:64] FLAG: --address="0.0.0.0" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794591 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794598 2567 flags.go:64] FLAG: --anonymous-auth="true" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794602 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794607 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794612 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794616 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794621 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794624 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794627 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794630 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794633 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794637 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794640 2567 flags.go:64] FLAG: --cgroup-root="" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794643 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794646 2567 flags.go:64] FLAG: --client-ca-file="" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794649 2567 flags.go:64] FLAG: --cloud-provider="external" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794652 2567 flags.go:64] FLAG: --cluster-dns="[]" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794655 2567 flags.go:64] FLAG: --cluster-domain="" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794658 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794661 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794664 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794667 2567 flags.go:64] FLAG: --container-log-max-files="5" Sep 05 02:17:55.799972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794671 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794674 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794676 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794679 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794682 2567 flags.go:64] FLAG: --contention-profiling="false" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794685 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794687 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794690 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794693 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794697 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794700 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794703 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794706 2567 flags.go:64] FLAG: --enable-load-reader="false" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794708 2567 flags.go:64] FLAG: --enable-server="true" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794713 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794717 2567 flags.go:64] FLAG: --event-burst="100" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794721 2567 flags.go:64] FLAG: --event-qps="50" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794723 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794726 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794729 2567 flags.go:64] FLAG: --eviction-hard="" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794732 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794736 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794739 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794742 2567 flags.go:64] FLAG: --eviction-soft="" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794745 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 05 02:17:55.800565 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794747 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794750 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794753 2567 flags.go:64] FLAG: --experimental-mounter-path="" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794756 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794759 2567 flags.go:64] FLAG: --fail-swap-on="true" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794762 2567 flags.go:64] FLAG: --feature-gates="" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794765 2567 flags.go:64] FLAG: --file-check-frequency="20s" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794768 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794771 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794774 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794777 2567 flags.go:64] FLAG: --healthz-port="10248" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794780 2567 flags.go:64] FLAG: --help="false" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794783 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-128-63.ec2.internal" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794786 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794788 2567 flags.go:64] FLAG: --http-check-frequency="20s" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794791 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794794 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794797 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794800 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794803 2567 flags.go:64] FLAG: --image-service-endpoint="" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794805 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794809 2567 flags.go:64] FLAG: --kube-api-burst="100" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794812 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794815 2567 flags.go:64] FLAG: --kube-api-qps="50" Sep 05 02:17:55.801193 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794818 2567 flags.go:64] FLAG: --kube-reserved="" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794821 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794824 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794826 2567 flags.go:64] FLAG: --kubelet-cgroups="" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794829 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794832 2567 flags.go:64] FLAG: --lock-file="" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794835 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794837 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794840 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794846 2567 flags.go:64] FLAG: --log-json-split-stream="false" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794848 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794851 2567 flags.go:64] FLAG: --log-text-split-stream="false" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794854 2567 flags.go:64] FLAG: --logging-format="text" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794856 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794860 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794862 2567 flags.go:64] FLAG: --manifest-url="" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794865 2567 flags.go:64] FLAG: --manifest-url-header="" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794870 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794873 2567 flags.go:64] FLAG: --max-open-files="1000000" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794877 2567 flags.go:64] FLAG: --max-pods="110" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794880 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794883 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794886 2567 flags.go:64] FLAG: --memory-manager-policy="None" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794889 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794892 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 05 02:17:55.801805 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794894 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794897 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794904 2567 flags.go:64] FLAG: --node-status-max-images="50" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794907 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794911 2567 flags.go:64] FLAG: --oom-score-adj="-999" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794914 2567 flags.go:64] FLAG: --pod-cidr="" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794917 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794923 2567 flags.go:64] FLAG: --pod-manifest-path="" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794926 2567 flags.go:64] FLAG: --pod-max-pids="-1" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794929 2567 flags.go:64] FLAG: --pods-per-core="0" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794932 2567 flags.go:64] FLAG: --port="10250" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794935 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794938 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e523803208e90701" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794941 2567 flags.go:64] FLAG: --qos-reserved="" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794943 2567 flags.go:64] FLAG: --read-only-port="10255" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794946 2567 flags.go:64] FLAG: --register-node="true" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794949 2567 flags.go:64] FLAG: --register-with-taints="" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794952 2567 flags.go:64] FLAG: --registry-burst="10" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794955 2567 flags.go:64] FLAG: --registry-qps="5" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794958 2567 flags.go:64] FLAG: --reserved-cpus="" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794960 2567 flags.go:64] FLAG: --reserved-memory="" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794964 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794966 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794969 2567 flags.go:64] FLAG: --rotate-certificates="false" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794972 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 05 02:17:55.802428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794974 2567 flags.go:64] FLAG: --runonce="false" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794977 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794980 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794983 2567 flags.go:64] FLAG: --seccomp-default="false" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794986 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794988 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794991 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794994 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794997 2567 flags.go:64] FLAG: --storage-driver-password="root" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.794999 2567 flags.go:64] FLAG: --storage-driver-secure="false" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795002 2567 flags.go:64] FLAG: --storage-driver-table="stats" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795006 2567 flags.go:64] FLAG: --storage-driver-user="root" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795009 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795011 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795015 2567 flags.go:64] FLAG: --system-cgroups="" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795017 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795022 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795028 2567 flags.go:64] FLAG: --tls-cert-file="" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795031 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795035 2567 flags.go:64] FLAG: --tls-min-version="" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795037 2567 flags.go:64] FLAG: --tls-private-key-file="" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795040 2567 flags.go:64] FLAG: --topology-manager-policy="none" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795043 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795045 2567 flags.go:64] FLAG: --topology-manager-scope="container" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795048 2567 flags.go:64] FLAG: --v="2" Sep 05 02:17:55.803107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795052 2567 flags.go:64] FLAG: --version="false" Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795056 2567 flags.go:64] FLAG: --vmodule="" Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795060 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.795063 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795171 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795175 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795178 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795180 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795183 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795185 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795187 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795190 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795192 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795195 2567 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795198 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795202 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795205 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795208 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795212 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795215 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 02:17:55.803725 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795217 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795220 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795223 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795225 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795230 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795233 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795236 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795239 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795242 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795245 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795248 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795250 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795253 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795256 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795258 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795261 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795263 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795266 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795268 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 02:17:55.804210 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795270 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795273 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795275 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795277 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795280 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795283 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795285 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795287 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795290 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795292 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795295 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795299 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795301 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795304 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795307 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795310 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795313 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795317 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795320 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795322 2567 feature_gate.go:328] unrecognized feature gate: Example Sep 05 02:17:55.804679 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795325 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795327 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795330 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795333 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795335 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795337 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795340 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795342 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795345 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795347 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795350 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795352 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795355 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795357 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795359 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795362 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795364 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795367 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795369 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795372 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 02:17:55.805197 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795374 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795376 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795378 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795382 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795385 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795387 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795390 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795392 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795394 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795399 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795401 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795404 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795406 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795409 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795411 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795414 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795416 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795419 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795421 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795424 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 02:17:55.805705 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795435 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795438 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795440 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.795443 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.796339 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.804334 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.804349 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.804365 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.804374 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804440 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804445 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804449 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804452 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804455 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804458 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 02:17:55.806196 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804461 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804463 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804466 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804469 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804471 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804474 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804483 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804486 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804489 2567 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804491 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804494 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804497 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804499 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804502 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804504 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804507 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804509 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804512 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804515 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804518 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 02:17:55.806651 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804520 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804523 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804526 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804528 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804531 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804534 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804536 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804539 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804541 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804544 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804546 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804549 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804551 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804553 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804556 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804558 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804561 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804563 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804565 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804592 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 02:17:55.807208 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804596 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804599 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804602 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804605 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804607 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804610 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804612 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804615 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804617 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804620 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804623 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804626 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804628 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804631 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804634 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804637 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804640 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804642 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804645 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804647 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 02:17:55.807696 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804649 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804652 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804654 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804658 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804662 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804665 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804669 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804673 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804675 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804678 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804683 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804686 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804697 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804699 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804702 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804705 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804707 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804709 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804712 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 02:17:55.808179 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804715 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804717 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804720 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804723 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804726 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804728 2567 feature_gate.go:328] unrecognized feature gate: Example Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804731 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804733 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804736 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804739 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804743 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804746 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804749 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804751 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.804756 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 05 02:17:55.808678 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804906 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804910 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804913 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804916 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804919 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804922 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804924 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804927 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804930 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804932 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804935 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804943 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804947 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804950 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804953 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804955 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804958 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804960 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804963 2567 feature_gate.go:328] unrecognized feature gate: Example Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804965 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 02:17:55.809041 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804968 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804971 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804973 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804976 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804978 2567 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804981 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804983 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804986 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804988 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804991 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804993 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804996 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.804998 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805000 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805003 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805005 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805008 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805010 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805012 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805015 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 02:17:55.809515 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805017 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805019 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805022 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805024 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805035 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805037 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805040 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805043 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805045 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805048 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805050 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805053 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805056 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805058 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805061 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805063 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805066 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805068 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805071 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 02:17:55.810046 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805073 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805076 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805078 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805080 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805083 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805085 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805088 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805090 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805093 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805095 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805098 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805100 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805102 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805105 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805107 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805110 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805112 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805114 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805122 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805125 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 02:17:55.810529 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805127 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805129 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805132 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805134 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805136 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805139 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805143 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805146 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805149 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805152 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805154 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805158 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805161 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805163 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805166 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805169 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805171 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805184 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805186 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 02:17:55.811120 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:55.805189 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 02:17:55.811601 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.805193 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 05 02:17:55.811601 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.805305 2567 server.go:962] "Client rotation is on, will bootstrap in background" Sep 05 02:17:55.811601 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.807806 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 05 02:17:55.811601 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.808824 2567 server.go:1019] "Starting client certificate rotation" Sep 05 02:17:55.811601 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.808953 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 05 02:17:55.811601 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.808992 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 05 02:17:55.811881 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.811866 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 05 02:17:55.814534 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.814516 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 05 02:17:55.827102 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.827081 2567 log.go:25] "Validated CRI v1 runtime API" Sep 05 02:17:55.830799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.830779 2567 log.go:25] "Validated CRI v1 image API" Sep 05 02:17:55.831697 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.831680 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 05 02:17:55.836585 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.836550 2567 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9c52858f-7705-4ee6-9af7-9998c59326a4:/dev/nvme0n1p4 b72bd0e3-f946-41d0-bfe7-9db3a32201be:/dev/nvme0n1p3] Sep 05 02:17:55.836636 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.836588 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 05 02:17:55.840628 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.840610 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 05 02:17:55.841921 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.841808 2567 manager.go:216] Machine: {Timestamp:2026-09-05 02:17:55.840330807 +0000 UTC m=+0.354109396 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104091 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c8cc5f2308d9564b9908bc196a022 SystemUUID:ec2c8cc5-f230-8d95-64b9-908bc196a022 BootID:b788778b-c034-44a1-aa11-cb7f295b98ef Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e4:a1:f8:7a:85 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e4:a1:f8:7a:85 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:62:a9:77:de:e1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 05 02:17:55.841921 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.841917 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 05 02:17:55.842024 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.841999 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 05 02:17:55.845641 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.845615 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 05 02:17:55.845786 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.845644 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-63.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 05 02:17:55.847203 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.847193 2567 topology_manager.go:138] "Creating topology manager with none policy" Sep 05 02:17:55.847242 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.847205 2567 container_manager_linux.go:309] "Creating device plugin manager" Sep 05 02:17:55.847242 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.847212 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 05 02:17:55.848001 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.847991 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 05 02:17:55.849855 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.849845 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 05 02:17:55.854238 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.854225 2567 state_mem.go:36] "Initialized new in-memory state store" Sep 05 02:17:55.854373 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.854363 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 05 02:17:55.854412 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.854398 2567 kubelet.go:486] "Attempting to sync node with API server" Sep 05 02:17:55.854412 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.854409 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 05 02:17:55.854467 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.854420 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 05 02:17:55.854467 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.854430 2567 kubelet.go:398] "Adding apiserver pod source" Sep 05 02:17:55.854467 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.854439 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 05 02:17:55.854803 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.854788 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 05 02:17:55.854904 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.854812 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 05 02:17:55.858636 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.858607 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 05 02:17:55.861691 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.861676 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 05 02:17:55.861751 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.861707 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 05 02:17:55.862682 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862670 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 05 02:17:55.862723 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862689 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 05 02:17:55.862723 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862695 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 05 02:17:55.862723 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862701 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 05 02:17:55.862723 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862710 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 05 02:17:55.862723 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862717 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 05 02:17:55.862851 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862758 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 05 02:17:55.862851 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862765 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 05 02:17:55.862851 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862772 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 05 02:17:55.862851 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862778 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 05 02:17:55.862851 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862787 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 05 02:17:55.862851 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.862792 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 05 02:17:55.863044 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.863001 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6r49m" Sep 05 02:17:55.864444 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.864434 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 05 02:17:55.864479 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.864444 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 05 02:17:55.865317 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:55.865300 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 05 02:17:55.865355 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:55.865301 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-63.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 05 02:17:55.868352 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.868339 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6r49m" Sep 05 02:17:55.868449 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.868437 2567 server.go:1268] "Started kubelet" Sep 05 02:17:55.868555 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.868523 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 05 02:17:55.868618 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.868544 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 05 02:17:55.868669 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.868647 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 05 02:17:55.869337 ip-10-0-128-63 systemd[1]: Started Kubernetes Kubelet. Sep 05 02:17:55.872309 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.872122 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 05 02:17:55.873472 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.873456 2567 server.go:310] "Adding debug handlers to kubelet server" Sep 05 02:17:55.873847 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:55.873822 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 05 02:17:55.875649 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.875629 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-63.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 05 02:17:55.875750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.875739 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 05 02:17:55.875809 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.875763 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 05 02:17:55.875932 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:55.875911 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:55.876543 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876526 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 05 02:17:55.876637 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876552 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 05 02:17:55.876637 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876565 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 05 02:17:55.876637 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876588 2567 factory.go:55] Registering systemd factory Sep 05 02:17:55.876637 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876603 2567 factory.go:223] Registration of the systemd container factory successfully Sep 05 02:17:55.876785 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876670 2567 reconstruct.go:97] "Volume reconstruction finished" Sep 05 02:17:55.876785 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876676 2567 reconciler.go:29] "Reconciler: start to sync state" Sep 05 02:17:55.876854 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876784 2567 factory.go:153] Registering CRI-O factory Sep 05 02:17:55.876854 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876795 2567 factory.go:223] Registration of the crio container factory successfully Sep 05 02:17:55.876854 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876839 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 05 02:17:55.876854 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876853 2567 factory.go:103] Registering Raw factory Sep 05 02:17:55.876998 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.876864 2567 manager.go:1196] Started watching for new ooms in manager Sep 05 02:17:55.877228 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.877216 2567 manager.go:349] Starting recovery of all containers Sep 05 02:17:55.885090 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.885074 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 02:17:55.887082 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.887060 2567 manager.go:354] Recovery completed Sep 05 02:17:55.887982 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:55.887958 2567 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-63.ec2.internal\" not found" node="ip-10-0-128-63.ec2.internal" Sep 05 02:17:55.891401 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.891388 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 02:17:55.894231 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.894210 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientMemory" Sep 05 02:17:55.894316 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.894247 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 02:17:55.894316 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.894259 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientPID" Sep 05 02:17:55.894927 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.894906 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 05 02:17:55.895000 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.894927 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 05 02:17:55.895000 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.894951 2567 state_mem.go:36] "Initialized new in-memory state store" Sep 05 02:17:55.895058 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.894950 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 02:17:55.896155 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.896140 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientMemory" Sep 05 02:17:55.896211 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.896167 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 02:17:55.896211 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.896177 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientPID" Sep 05 02:17:55.897036 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.897025 2567 policy_none.go:49] "None policy: Start" Sep 05 02:17:55.897113 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.897041 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 05 02:17:55.897113 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.897051 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 05 02:17:55.898897 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.898887 2567 policy_none.go:47] "Start" Sep 05 02:17:55.933112 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.933092 2567 manager.go:337] "Starting Device Plugin manager" Sep 05 02:17:55.933214 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:55.933134 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 05 02:17:55.933214 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.933144 2567 server.go:87] "Starting device plugin registration server" Sep 05 02:17:55.933561 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.933545 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 05 02:17:55.933822 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.933590 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 05 02:17:55.933822 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.933753 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 05 02:17:55.934065 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.933868 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 05 02:17:55.934065 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.933877 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 05 02:17:55.934360 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:55.934333 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 05 02:17:55.934436 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:55.934393 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:55.982588 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.982460 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 05 02:17:55.983827 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.983809 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 05 02:17:55.983890 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.983857 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 05 02:17:55.983890 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.983881 2567 kubelet.go:2448] "Starting kubelet main sync loop" Sep 05 02:17:55.983948 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:55.983921 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 05 02:17:55.986933 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:55.986911 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 02:17:56.034238 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.034207 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 02:17:56.035279 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.035263 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientMemory" Sep 05 02:17:56.035374 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.035295 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 02:17:56.035374 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.035306 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientPID" Sep 05 02:17:56.035374 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.035330 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.042357 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.042334 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.042402 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.042367 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-63.ec2.internal\": node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:56.060040 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.060013 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:56.083988 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.083964 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal"] Sep 05 02:17:56.084041 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.084031 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 02:17:56.085540 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.085525 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientMemory" Sep 05 02:17:56.085614 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.085554 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 02:17:56.085614 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.085587 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientPID" Sep 05 02:17:56.087716 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.087704 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 02:17:56.087838 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.087824 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.087870 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.087856 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 02:17:56.088502 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.088488 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientMemory" Sep 05 02:17:56.088605 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.088514 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 02:17:56.088605 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.088528 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientPID" Sep 05 02:17:56.088605 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.088512 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientMemory" Sep 05 02:17:56.088731 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.088608 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 02:17:56.088731 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.088619 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientPID" Sep 05 02:17:56.090779 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.090764 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.090868 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.090790 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 02:17:56.091665 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.091650 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientMemory" Sep 05 02:17:56.091750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.091676 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 02:17:56.091750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.091691 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeHasSufficientPID" Sep 05 02:17:56.120385 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.120349 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-63.ec2.internal\" not found" node="ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.125044 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.125025 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-63.ec2.internal\" not found" node="ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.160136 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.160106 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:56.178635 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.178608 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e20bfcd8fa7c3756f9a221829a74dd57-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal\" (UID: \"e20bfcd8fa7c3756f9a221829a74dd57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.178698 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.178640 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e20bfcd8fa7c3756f9a221829a74dd57-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal\" (UID: \"e20bfcd8fa7c3756f9a221829a74dd57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.178698 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.178659 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/07df7873c55aad720a5c1b311be3ff60-config\") pod \"kube-apiserver-proxy-ip-10-0-128-63.ec2.internal\" (UID: \"07df7873c55aad720a5c1b311be3ff60\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.260554 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.260494 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:56.278794 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.278767 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e20bfcd8fa7c3756f9a221829a74dd57-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal\" (UID: \"e20bfcd8fa7c3756f9a221829a74dd57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.278862 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.278792 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e20bfcd8fa7c3756f9a221829a74dd57-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal\" (UID: \"e20bfcd8fa7c3756f9a221829a74dd57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.278862 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.278811 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e20bfcd8fa7c3756f9a221829a74dd57-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal\" (UID: \"e20bfcd8fa7c3756f9a221829a74dd57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.278862 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.278837 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/07df7873c55aad720a5c1b311be3ff60-config\") pod \"kube-apiserver-proxy-ip-10-0-128-63.ec2.internal\" (UID: \"07df7873c55aad720a5c1b311be3ff60\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.278945 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.278873 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/07df7873c55aad720a5c1b311be3ff60-config\") pod \"kube-apiserver-proxy-ip-10-0-128-63.ec2.internal\" (UID: \"07df7873c55aad720a5c1b311be3ff60\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.278945 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.278865 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e20bfcd8fa7c3756f9a221829a74dd57-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal\" (UID: \"e20bfcd8fa7c3756f9a221829a74dd57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.361190 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.361150 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:56.422739 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.422709 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.427498 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.427306 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.461531 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.461497 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:56.562094 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.562008 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:56.662604 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.662561 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:56.714256 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.714224 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 02:17:56.763517 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:56.763492 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-63.ec2.internal\" not found" Sep 05 02:17:56.808999 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.808960 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 05 02:17:56.809401 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.809128 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 05 02:17:56.809401 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.809167 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 05 02:17:56.809401 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.809187 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 05 02:17:56.859038 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.858956 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 02:17:56.874244 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.874198 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-04 02:12:56 +0000 UTC" deadline="2028-05-16 07:22:25.121080348 +0000 UTC" Sep 05 02:17:56.874244 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.874239 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14861h4m28.246844611s" Sep 05 02:17:56.876382 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.876364 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.876382 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.876350 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 05 02:17:56.884987 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.884965 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 05 02:17:56.886113 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.886100 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" Sep 05 02:17:56.890639 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.890621 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 05 02:17:56.896069 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.896051 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 05 02:17:56.908854 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.908834 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9thlh" Sep 05 02:17:56.919305 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:56.919283 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9thlh" Sep 05 02:17:57.124132 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:57.124087 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20bfcd8fa7c3756f9a221829a74dd57.slice/crio-ea7ca4ae011c4dd57b7291473e09ac7c56a58e929cef6773aae5c78dcd808708 WatchSource:0}: Error finding container ea7ca4ae011c4dd57b7291473e09ac7c56a58e929cef6773aae5c78dcd808708: Status 404 returned error can't find the container with id ea7ca4ae011c4dd57b7291473e09ac7c56a58e929cef6773aae5c78dcd808708 Sep 05 02:17:57.124436 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:17:57.124421 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07df7873c55aad720a5c1b311be3ff60.slice/crio-e05b5e1652a11b82d190cbf8d29e40814716d09b0bc3dfec611d2738c37df2f0 WatchSource:0}: Error finding container e05b5e1652a11b82d190cbf8d29e40814716d09b0bc3dfec611d2738c37df2f0: Status 404 returned error can't find the container with id e05b5e1652a11b82d190cbf8d29e40814716d09b0bc3dfec611d2738c37df2f0 Sep 05 02:17:57.129590 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:57.129558 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 05 02:17:57.700190 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:57.700159 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 02:17:58.883844 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.883802 2567 apiserver.go:52] "Watching apiserver" Sep 05 02:17:58.893757 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.893722 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 05 02:17:58.894880 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.894851 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-r47jj","kube-system/global-pull-secret-syncer-74l8j","kube-system/konnectivity-agent-zgb7b","openshift-dns/node-resolver-p4p4n","openshift-image-registry/node-ca-qx5mt","openshift-multus/multus-additional-cni-plugins-5q89w","openshift-network-diagnostics/network-check-target-jslqd","openshift-network-operator/iptables-alerter-rrpll","openshift-ovn-kubernetes/ovnkube-node-8cc78","kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6","openshift-cluster-node-tuning-operator/tuned-5fc7x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal","openshift-multus/multus-5bshb"] Sep 05 02:17:58.897342 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.897314 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:17:58.897456 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:58.897406 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:17:58.899780 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.899755 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:58.899909 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:58.899837 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:17:58.902229 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.902206 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:17:58.904457 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.904429 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zpjmv\"" Sep 05 02:17:58.904674 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.904657 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 05 02:17:58.904874 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.904852 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 05 02:17:58.905651 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.905441 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:58.907818 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.907797 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 05 02:17:58.907965 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.907798 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5pdfj\"" Sep 05 02:17:58.908061 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.908016 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 05 02:17:58.909612 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.909594 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:58.911927 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.911902 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 05 02:17:58.912022 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.911998 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mfwkn\"" Sep 05 02:17:58.912124 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.912108 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 05 02:17:58.912166 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.912108 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 05 02:17:58.913319 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.913299 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:58.914733 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.914706 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e6dbc91-2b3f-40a1-9b30-525617d8c100-tmp-dir\") pod \"node-resolver-p4p4n\" (UID: \"9e6dbc91-2b3f-40a1-9b30-525617d8c100\") " pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:58.914845 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.914746 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:17:58.914845 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.914779 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd04d4e5-d457-482a-b1ee-a2d606c18b81-konnectivity-ca\") pod \"konnectivity-agent-zgb7b\" (UID: \"fd04d4e5-d457-482a-b1ee-a2d606c18b81\") " pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:17:58.914845 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.914811 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9e6dbc91-2b3f-40a1-9b30-525617d8c100-hosts-file\") pod \"node-resolver-p4p4n\" (UID: \"9e6dbc91-2b3f-40a1-9b30-525617d8c100\") " pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:58.914845 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.914837 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5krg7\" (UniqueName: \"kubernetes.io/projected/9e6dbc91-2b3f-40a1-9b30-525617d8c100-kube-api-access-5krg7\") pod \"node-resolver-p4p4n\" (UID: \"9e6dbc91-2b3f-40a1-9b30-525617d8c100\") " pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:58.915042 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.914863 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jch88\" (UniqueName: \"kubernetes.io/projected/257406f0-bf02-4fea-a10b-daf9750ddefc-kube-api-access-jch88\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:17:58.915042 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.914888 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d398890b-b9a3-4b54-abdd-d0f90f315b66-kubelet-config\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:58.915042 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.914932 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d398890b-b9a3-4b54-abdd-d0f90f315b66-dbus\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:58.915042 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.914987 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:58.915042 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.915036 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd04d4e5-d457-482a-b1ee-a2d606c18b81-agent-certs\") pod \"konnectivity-agent-zgb7b\" (UID: \"fd04d4e5-d457-482a-b1ee-a2d606c18b81\") " pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:17:58.916930 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.916890 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 05 02:17:58.917211 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.917190 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4q4hj\"" Sep 05 02:17:58.917407 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.917393 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 05 02:17:58.917593 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.917580 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 05 02:17:58.917868 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.917855 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 05 02:17:58.918051 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.918038 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 05 02:17:58.921265 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.921227 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:17:58.921381 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:58.921310 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:17:58.924014 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.923985 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:58.926893 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.926362 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 05 02:17:58.926893 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.926429 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 05 02:17:58.926893 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.926547 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bxmp5\"" Sep 05 02:17:58.926893 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.926662 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 05 02:17:58.927611 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.927308 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:58.929175 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.929154 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:58.930944 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.930924 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 05 02:17:58.931052 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.931037 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 05 02:17:58.931115 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.931073 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 05 02:17:58.931586 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.931557 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vckmw\"" Sep 05 02:17:58.931818 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.931803 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 05 02:17:58.932041 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.932026 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 05 02:17:58.934006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.933610 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 05 02:17:58.934006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.933827 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 05 02:17:58.934006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.933975 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 05 02:17:58.934201 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.934152 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2sxtz\"" Sep 05 02:17:58.934726 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.934550 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 05 02:17:58.934819 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.934804 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:58.938181 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.937614 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 05 02:17:58.938181 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.937748 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5bshb" Sep 05 02:17:58.938181 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.937821 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tqddg\"" Sep 05 02:17:58.938181 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.937904 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 05 02:17:58.940057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.940040 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 05 02:17:58.940205 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.940183 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l74gn\"" Sep 05 02:17:58.950881 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.950829 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 02:12:57 +0000 UTC" deadline="2028-05-18 03:14:08.968073268 +0000 UTC" Sep 05 02:17:58.950881 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:58.950879 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14904h56m10.017197962s" Sep 05 02:17:59.005662 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.005638 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015392 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9e6dbc91-2b3f-40a1-9b30-525617d8c100-hosts-file\") pod \"node-resolver-p4p4n\" (UID: \"9e6dbc91-2b3f-40a1-9b30-525617d8c100\") " pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015510 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015541 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-modprobe-d\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015567 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-socket-dir-parent\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015539 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9e6dbc91-2b3f-40a1-9b30-525617d8c100-hosts-file\") pod \"node-resolver-p4p4n\" (UID: \"9e6dbc91-2b3f-40a1-9b30-525617d8c100\") " pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015593 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-system-cni-dir\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015619 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jch88\" (UniqueName: \"kubernetes.io/projected/257406f0-bf02-4fea-a10b-daf9750ddefc-kube-api-access-jch88\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015645 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmkn2\" (UniqueName: \"kubernetes.io/projected/687f9036-0a6d-4fb4-abef-d64801928cc1-kube-api-access-lmkn2\") pod \"node-ca-qx5mt\" (UID: \"687f9036-0a6d-4fb4-abef-d64801928cc1\") " pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015670 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75cd35e8-1c2c-4bc6-a121-15adc834eb69-system-cni-dir\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015695 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75cd35e8-1c2c-4bc6-a121-15adc834eb69-cni-binary-copy\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015718 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-systemd-units\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015740 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-kubernetes\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015766 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-sysctl-conf\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015803 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e6dbc91-2b3f-40a1-9b30-525617d8c100-tmp-dir\") pod \"node-resolver-p4p4n\" (UID: \"9e6dbc91-2b3f-40a1-9b30-525617d8c100\") " pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015826 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75cd35e8-1c2c-4bc6-a121-15adc834eb69-cnibin\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.016799 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015850 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stpgx\" (UniqueName: \"kubernetes.io/projected/75cd35e8-1c2c-4bc6-a121-15adc834eb69-kube-api-access-stpgx\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015901 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3132f740-a0ad-4ea5-be63-aaff35aa174f-host-slash\") pod \"iptables-alerter-rrpll\" (UID: \"3132f740-a0ad-4ea5-be63-aaff35aa174f\") " pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.015948 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-systemd\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016002 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-lib-modules\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016037 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-run-k8s-cni-cncf-io\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016061 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-sysconfig\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016096 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d398890b-b9a3-4b54-abdd-d0f90f315b66-kubelet-config\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d398890b-b9a3-4b54-abdd-d0f90f315b66-dbus\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016146 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016201 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/687f9036-0a6d-4fb4-abef-d64801928cc1-host\") pod \"node-ca-qx5mt\" (UID: \"687f9036-0a6d-4fb4-abef-d64801928cc1\") " pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016223 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-slash\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016247 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-run-ovn\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-tuned\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016295 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/75cd35e8-1c2c-4bc6-a121-15adc834eb69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016319 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-run-openvswitch\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016343 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/306ce2c3-da47-4ecd-965a-5d2bda450872-ovnkube-config\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016366 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-registration-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016389 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-var-lib-kubelet\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016388 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e6dbc91-2b3f-40a1-9b30-525617d8c100-tmp-dir\") pod \"node-resolver-p4p4n\" (UID: \"9e6dbc91-2b3f-40a1-9b30-525617d8c100\") " pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016415 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-var-lib-cni-bin\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016444 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75cd35e8-1c2c-4bc6-a121-15adc834eb69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016479 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016555 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-socket-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016586 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-device-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016612 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016634 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-tmp\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016676 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d398890b-b9a3-4b54-abdd-d0f90f315b66-kubelet-config\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016698 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d398890b-b9a3-4b54-abdd-d0f90f315b66-dbus\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.016710 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016743 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/687f9036-0a6d-4fb4-abef-d64801928cc1-serviceca\") pod \"node-ca-qx5mt\" (UID: \"687f9036-0a6d-4fb4-abef-d64801928cc1\") " pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.016791 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret podName:d398890b-b9a3-4b54-abdd-d0f90f315b66 nodeName:}" failed. No retries permitted until 2026-09-05 02:17:59.516767733 +0000 UTC m=+3.002626701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret") pod "global-pull-secret-syncer-74l8j" (UID: "d398890b-b9a3-4b54-abdd-d0f90f315b66") : object "kube-system"/"original-pull-secret" not registered Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016813 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-run-netns\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.018386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016854 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/306ce2c3-da47-4ecd-965a-5d2bda450872-ovn-node-metrics-cert\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016893 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krg7\" (UniqueName: \"kubernetes.io/projected/9e6dbc91-2b3f-40a1-9b30-525617d8c100-kube-api-access-5krg7\") pod \"node-resolver-p4p4n\" (UID: \"9e6dbc91-2b3f-40a1-9b30-525617d8c100\") " pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.016953 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-run-systemd\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017058 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-etc-openvswitch\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017084 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-sysctl-d\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017109 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-cni-binary-copy\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017151 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-log-socket\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017185 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-run-ovn-kubernetes\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017230 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-run\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017262 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-run-netns\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017291 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-hostroot\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017316 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-cni-bin\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017354 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-cni-netd\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017382 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/306ce2c3-da47-4ecd-965a-5d2bda450872-ovnkube-script-lib\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017406 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8v96\" (UniqueName: \"kubernetes.io/projected/306ce2c3-da47-4ecd-965a-5d2bda450872-kube-api-access-q8v96\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017429 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017451 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-cni-dir\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.019232 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-cnibin\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017540 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-conf-dir\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017573 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-var-lib-openvswitch\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017605 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-daemon-config\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017630 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-run-multus-certs\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017677 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd04d4e5-d457-482a-b1ee-a2d606c18b81-konnectivity-ca\") pod \"konnectivity-agent-zgb7b\" (UID: \"fd04d4e5-d457-482a-b1ee-a2d606c18b81\") " pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017701 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3132f740-a0ad-4ea5-be63-aaff35aa174f-iptables-alerter-script\") pod \"iptables-alerter-rrpll\" (UID: \"3132f740-a0ad-4ea5-be63-aaff35aa174f\") " pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017724 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrrp2\" (UniqueName: \"kubernetes.io/projected/3132f740-a0ad-4ea5-be63-aaff35aa174f-kube-api-access-rrrp2\") pod \"iptables-alerter-rrpll\" (UID: \"3132f740-a0ad-4ea5-be63-aaff35aa174f\") " pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017750 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-node-log\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017774 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-sys-fs\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017819 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-sys\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017844 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-var-lib-cni-multus\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017867 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-var-lib-kubelet\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017893 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw9p7\" (UniqueName: \"kubernetes.io/projected/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-kube-api-access-sw9p7\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017915 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-etc-kubernetes\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.020116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017938 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bjht\" (UniqueName: \"kubernetes.io/projected/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-kube-api-access-8bjht\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.017960 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-kubelet\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.018001 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd04d4e5-d457-482a-b1ee-a2d606c18b81-agent-certs\") pod \"konnectivity-agent-zgb7b\" (UID: \"fd04d4e5-d457-482a-b1ee-a2d606c18b81\") " pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.018025 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/75cd35e8-1c2c-4bc6-a121-15adc834eb69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.018047 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/306ce2c3-da47-4ecd-965a-5d2bda450872-env-overrides\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.018068 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-host\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.018090 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dhf5\" (UniqueName: \"kubernetes.io/projected/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-kube-api-access-7dhf5\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.018310 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.018379 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs podName:257406f0-bf02-4fea-a10b-daf9750ddefc nodeName:}" failed. No retries permitted until 2026-09-05 02:17:59.518361454 +0000 UTC m=+3.004220414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs") pod "network-metrics-daemon-r47jj" (UID: "257406f0-bf02-4fea-a10b-daf9750ddefc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.018847 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd04d4e5-d457-482a-b1ee-a2d606c18b81-konnectivity-ca\") pod \"konnectivity-agent-zgb7b\" (UID: \"fd04d4e5-d457-482a-b1ee-a2d606c18b81\") " pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:17:59.021003 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.019067 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 05 02:17:59.021535 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.021469 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal" event={"ID":"07df7873c55aad720a5c1b311be3ff60","Type":"ContainerStarted","Data":"e05b5e1652a11b82d190cbf8d29e40814716d09b0bc3dfec611d2738c37df2f0"} Sep 05 02:17:59.024011 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.023948 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd04d4e5-d457-482a-b1ee-a2d606c18b81-agent-certs\") pod \"konnectivity-agent-zgb7b\" (UID: \"fd04d4e5-d457-482a-b1ee-a2d606c18b81\") " pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:17:59.024197 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.024165 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" event={"ID":"e20bfcd8fa7c3756f9a221829a74dd57","Type":"ContainerStarted","Data":"ea7ca4ae011c4dd57b7291473e09ac7c56a58e929cef6773aae5c78dcd808708"} Sep 05 02:17:59.030735 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.030706 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5krg7\" (UniqueName: \"kubernetes.io/projected/9e6dbc91-2b3f-40a1-9b30-525617d8c100-kube-api-access-5krg7\") pod \"node-resolver-p4p4n\" (UID: \"9e6dbc91-2b3f-40a1-9b30-525617d8c100\") " pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:59.032261 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.032225 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jch88\" (UniqueName: \"kubernetes.io/projected/257406f0-bf02-4fea-a10b-daf9750ddefc-kube-api-access-jch88\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:17:59.061650 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.061432 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 02:17:59.118956 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.118923 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmkn2\" (UniqueName: \"kubernetes.io/projected/687f9036-0a6d-4fb4-abef-d64801928cc1-kube-api-access-lmkn2\") pod \"node-ca-qx5mt\" (UID: \"687f9036-0a6d-4fb4-abef-d64801928cc1\") " pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.119134 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.118970 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75cd35e8-1c2c-4bc6-a121-15adc834eb69-system-cni-dir\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.119134 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.118992 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75cd35e8-1c2c-4bc6-a121-15adc834eb69-cni-binary-copy\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.119134 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119016 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-systemd-units\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.119134 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119034 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-kubernetes\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.119134 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119048 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-sysctl-conf\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.119134 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119066 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75cd35e8-1c2c-4bc6-a121-15adc834eb69-cnibin\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.119134 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119061 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75cd35e8-1c2c-4bc6-a121-15adc834eb69-system-cni-dir\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.119134 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119083 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stpgx\" (UniqueName: \"kubernetes.io/projected/75cd35e8-1c2c-4bc6-a121-15adc834eb69-kube-api-access-stpgx\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.119134 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3132f740-a0ad-4ea5-be63-aaff35aa174f-host-slash\") pod \"iptables-alerter-rrpll\" (UID: \"3132f740-a0ad-4ea5-be63-aaff35aa174f\") " pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119145 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-systemd\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119169 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-lib-modules\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119195 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3132f740-a0ad-4ea5-be63-aaff35aa174f-host-slash\") pod \"iptables-alerter-rrpll\" (UID: \"3132f740-a0ad-4ea5-be63-aaff35aa174f\") " pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119196 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-run-k8s-cni-cncf-io\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119229 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-run-k8s-cni-cncf-io\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119240 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-sysconfig\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119268 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/687f9036-0a6d-4fb4-abef-d64801928cc1-host\") pod \"node-ca-qx5mt\" (UID: \"687f9036-0a6d-4fb4-abef-d64801928cc1\") " pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119283 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-slash\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119302 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-run-ovn\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119325 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-tuned\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119340 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/75cd35e8-1c2c-4bc6-a121-15adc834eb69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119356 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-run-openvswitch\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119371 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/306ce2c3-da47-4ecd-965a-5d2bda450872-ovnkube-config\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119403 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-registration-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119409 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-sysconfig\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119428 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-var-lib-kubelet\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119448 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/687f9036-0a6d-4fb4-abef-d64801928cc1-host\") pod \"node-ca-qx5mt\" (UID: \"687f9036-0a6d-4fb4-abef-d64801928cc1\") " pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.119642 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-var-lib-cni-bin\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119475 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75cd35e8-1c2c-4bc6-a121-15adc834eb69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119485 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-slash\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119371 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-lib-modules\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119548 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-socket-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119574 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-device-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119581 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-run-ovn\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119597 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119621 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-tmp\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119651 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/687f9036-0a6d-4fb4-abef-d64801928cc1-serviceca\") pod \"node-ca-qx5mt\" (UID: \"687f9036-0a6d-4fb4-abef-d64801928cc1\") " pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119673 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-run-netns\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119697 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/306ce2c3-da47-4ecd-965a-5d2bda450872-ovn-node-metrics-cert\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119723 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-run-systemd\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119746 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-etc-openvswitch\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119769 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-sysctl-d\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119774 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75cd35e8-1c2c-4bc6-a121-15adc834eb69-cni-binary-copy\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.120294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119795 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-cni-binary-copy\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119824 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-log-socket\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119838 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-systemd-units\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119856 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-run-ovn-kubernetes\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119879 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-run\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119900 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-run-netns\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119903 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-kubernetes\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119923 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-hostroot\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119930 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-run-openvswitch\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119948 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-cni-bin\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119972 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-cni-netd\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120006 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/306ce2c3-da47-4ecd-965a-5d2bda450872-ovnkube-script-lib\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120022 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-sysctl-conf\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120024 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8v96\" (UniqueName: \"kubernetes.io/projected/306ce2c3-da47-4ecd-965a-5d2bda450872-kube-api-access-q8v96\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120064 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120092 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-cni-dir\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120116 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-cnibin\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120141 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-conf-dir\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.120907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120166 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-var-lib-openvswitch\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120191 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-daemon-config\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120203 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-registration-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.119281 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-systemd\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120217 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-run-multus-certs\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120241 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-var-lib-kubelet\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3132f740-a0ad-4ea5-be63-aaff35aa174f-iptables-alerter-script\") pod \"iptables-alerter-rrpll\" (UID: \"3132f740-a0ad-4ea5-be63-aaff35aa174f\") " pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120266 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-var-lib-cni-bin\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120272 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrrp2\" (UniqueName: \"kubernetes.io/projected/3132f740-a0ad-4ea5-be63-aaff35aa174f-kube-api-access-rrrp2\") pod \"iptables-alerter-rrpll\" (UID: \"3132f740-a0ad-4ea5-be63-aaff35aa174f\") " pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120301 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-node-log\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120326 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-sys-fs\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120362 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-sys\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120375 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75cd35e8-1c2c-4bc6-a121-15adc834eb69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120387 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-var-lib-cni-multus\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120388 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/75cd35e8-1c2c-4bc6-a121-15adc834eb69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120403 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120412 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-var-lib-kubelet\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.121550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120438 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw9p7\" (UniqueName: \"kubernetes.io/projected/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-kube-api-access-sw9p7\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120460 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-socket-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120463 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-etc-kubernetes\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120510 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-log-socket\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120488 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bjht\" (UniqueName: \"kubernetes.io/projected/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-kube-api-access-8bjht\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120545 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-kubelet\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/75cd35e8-1c2c-4bc6-a121-15adc834eb69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120579 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/306ce2c3-da47-4ecd-965a-5d2bda450872-env-overrides\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120594 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-host\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120608 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dhf5\" (UniqueName: \"kubernetes.io/projected/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-kube-api-access-7dhf5\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120626 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120642 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-modprobe-d\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120656 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-socket-dir-parent\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120672 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-system-cni-dir\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120728 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75cd35e8-1c2c-4bc6-a121-15adc834eb69-cnibin\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120736 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-system-cni-dir\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120771 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.122150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120784 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-run-ovn-kubernetes\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120808 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-cni-dir\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120829 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-run\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120835 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-cnibin\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120862 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-conf-dir\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120872 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-hostroot\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120889 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-run-netns\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120909 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-cni-bin\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120917 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-var-lib-openvswitch\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120952 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-cni-netd\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120989 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-run-multus-certs\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121447 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-node-log\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121465 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/687f9036-0a6d-4fb4-abef-d64801928cc1-serviceca\") pod \"node-ca-qx5mt\" (UID: \"687f9036-0a6d-4fb4-abef-d64801928cc1\") " pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121537 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-sys-fs\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121547 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-run-netns\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121591 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-sys\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121636 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-var-lib-cni-multus\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121681 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-host-var-lib-kubelet\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.122750 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121790 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/306ce2c3-da47-4ecd-965a-5d2bda450872-ovnkube-script-lib\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121847 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-host-kubelet\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121916 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3132f740-a0ad-4ea5-be63-aaff35aa174f-iptables-alerter-script\") pod \"iptables-alerter-rrpll\" (UID: \"3132f740-a0ad-4ea5-be63-aaff35aa174f\") " pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.121990 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-etc-kubernetes\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122210 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-cni-binary-copy\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.120464 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-device-dir\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122266 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/75cd35e8-1c2c-4bc6-a121-15adc834eb69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122344 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122592 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-host\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122672 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/306ce2c3-da47-4ecd-965a-5d2bda450872-env-overrides\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122772 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-modprobe-d\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122820 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-socket-dir-parent\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122819 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-etc-openvswitch\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122859 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/306ce2c3-da47-4ecd-965a-5d2bda450872-run-systemd\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.122968 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-sysctl-d\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.123222 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-multus-daemon-config\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.123484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.123313 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-tmp\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.124294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.123592 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/306ce2c3-da47-4ecd-965a-5d2bda450872-ovnkube-config\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.124294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.123911 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-etc-tuned\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.125180 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.125159 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/306ce2c3-da47-4ecd-965a-5d2bda450872-ovn-node-metrics-cert\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.129649 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.128930 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 02:17:59.129649 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.128982 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 02:17:59.129649 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.128997 2567 projected.go:196] Error preparing data for projected volume kube-api-access-th29q for pod openshift-network-diagnostics/network-check-target-jslqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:17:59.129649 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.129070 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q podName:b5be9e27-b6e4-4477-9c67-2245faef62e9 nodeName:}" failed. No retries permitted until 2026-09-05 02:17:59.629050908 +0000 UTC m=+3.114909899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-th29q" (UniqueName: "kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q") pod "network-check-target-jslqd" (UID: "b5be9e27-b6e4-4477-9c67-2245faef62e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:17:59.132953 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.132931 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dhf5\" (UniqueName: \"kubernetes.io/projected/a4cb2d0a-0b28-4d06-a48f-ce729554a02e-kube-api-access-7dhf5\") pod \"tuned-5fc7x\" (UID: \"a4cb2d0a-0b28-4d06-a48f-ce729554a02e\") " pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.134087 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.134066 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stpgx\" (UniqueName: \"kubernetes.io/projected/75cd35e8-1c2c-4bc6-a121-15adc834eb69-kube-api-access-stpgx\") pod \"multus-additional-cni-plugins-5q89w\" (UID: \"75cd35e8-1c2c-4bc6-a121-15adc834eb69\") " pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.134227 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.134111 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmkn2\" (UniqueName: \"kubernetes.io/projected/687f9036-0a6d-4fb4-abef-d64801928cc1-kube-api-access-lmkn2\") pod \"node-ca-qx5mt\" (UID: \"687f9036-0a6d-4fb4-abef-d64801928cc1\") " pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.134227 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.134118 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw9p7\" (UniqueName: \"kubernetes.io/projected/4240570d-29f2-4cf0-b8f3-7e702ad70eaa-kube-api-access-sw9p7\") pod \"aws-ebs-csi-driver-node-vb9c6\" (UID: \"4240570d-29f2-4cf0-b8f3-7e702ad70eaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.134437 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.134423 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrrp2\" (UniqueName: \"kubernetes.io/projected/3132f740-a0ad-4ea5-be63-aaff35aa174f-kube-api-access-rrrp2\") pod \"iptables-alerter-rrpll\" (UID: \"3132f740-a0ad-4ea5-be63-aaff35aa174f\") " pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.134788 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.134755 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bjht\" (UniqueName: \"kubernetes.io/projected/7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb-kube-api-access-8bjht\") pod \"multus-5bshb\" (UID: \"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb\") " pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.136732 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.136715 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8v96\" (UniqueName: \"kubernetes.io/projected/306ce2c3-da47-4ecd-965a-5d2bda450872-kube-api-access-q8v96\") pod \"ovnkube-node-8cc78\" (UID: \"306ce2c3-da47-4ecd-965a-5d2bda450872\") " pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.216203 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.216162 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:17:59.230997 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.230959 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p4p4n" Sep 05 02:17:59.243038 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.243011 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qx5mt" Sep 05 02:17:59.251918 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.251895 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5q89w" Sep 05 02:17:59.258754 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.258731 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rrpll" Sep 05 02:17:59.266557 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.266534 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:17:59.273388 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.273368 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" Sep 05 02:17:59.280081 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.280059 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" Sep 05 02:17:59.285783 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.285764 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5bshb" Sep 05 02:17:59.523320 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.523232 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:17:59.523320 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.523299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:17:59.523536 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.523354 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 02:17:59.523536 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.523408 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:17:59.523536 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.523413 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret podName:d398890b-b9a3-4b54-abdd-d0f90f315b66 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:00.523395836 +0000 UTC m=+4.009254802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret") pod "global-pull-secret-syncer-74l8j" (UID: "d398890b-b9a3-4b54-abdd-d0f90f315b66") : object "kube-system"/"original-pull-secret" not registered Sep 05 02:17:59.523536 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.523467 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs podName:257406f0-bf02-4fea-a10b-daf9750ddefc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:00.523455531 +0000 UTC m=+4.009314484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs") pod "network-metrics-daemon-r47jj" (UID: "257406f0-bf02-4fea-a10b-daf9750ddefc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:17:59.724378 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.724346 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:17:59.724550 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.724464 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 02:17:59.724550 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.724478 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 02:17:59.724550 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.724487 2567 projected.go:196] Error preparing data for projected volume kube-api-access-th29q for pod openshift-network-diagnostics/network-check-target-jslqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:17:59.724672 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:17:59.724551 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q podName:b5be9e27-b6e4-4477-9c67-2245faef62e9 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:00.724537501 +0000 UTC m=+4.210396453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-th29q" (UniqueName: "kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q") pod "network-check-target-jslqd" (UID: "b5be9e27-b6e4-4477-9c67-2245faef62e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:17:59.951518 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.951412 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 02:12:57 +0000 UTC" deadline="2028-05-25 17:30:52.297677682 +0000 UTC" Sep 05 02:17:59.951518 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:17:59.951461 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15087h12m52.346220265s" Sep 05 02:18:00.119707 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:00.119675 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4cb2d0a_0b28_4d06_a48f_ce729554a02e.slice/crio-99f456b5cf1a21674695a2f735b17c3bc7f42e9b03a1573f079c7428f20c117d WatchSource:0}: Error finding container 99f456b5cf1a21674695a2f735b17c3bc7f42e9b03a1573f079c7428f20c117d: Status 404 returned error can't find the container with id 99f456b5cf1a21674695a2f735b17c3bc7f42e9b03a1573f079c7428f20c117d Sep 05 02:18:00.124169 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:00.124148 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3132f740_a0ad_4ea5_be63_aaff35aa174f.slice/crio-e2b4cecdaaa2579c9b2be98273e71cf1df1cb0f3dc986c39f0b867337329b579 WatchSource:0}: Error finding container e2b4cecdaaa2579c9b2be98273e71cf1df1cb0f3dc986c39f0b867337329b579: Status 404 returned error can't find the container with id e2b4cecdaaa2579c9b2be98273e71cf1df1cb0f3dc986c39f0b867337329b579 Sep 05 02:18:00.125057 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:00.125037 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a211ccd_3d23_4de1_aa38_b4a3d2b3c4bb.slice/crio-e0933cb466f066a5c68904ec926e859d409a92ae178d2f2621a66c7d79fa2f76 WatchSource:0}: Error finding container e0933cb466f066a5c68904ec926e859d409a92ae178d2f2621a66c7d79fa2f76: Status 404 returned error can't find the container with id e0933cb466f066a5c68904ec926e859d409a92ae178d2f2621a66c7d79fa2f76 Sep 05 02:18:00.125864 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:00.125842 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod687f9036_0a6d_4fb4_abef_d64801928cc1.slice/crio-6f3bf79b6459c185b7fe24bd3c2fa9ad1c9f4e96de8d4f0be677e3924fd90ac3 WatchSource:0}: Error finding container 6f3bf79b6459c185b7fe24bd3c2fa9ad1c9f4e96de8d4f0be677e3924fd90ac3: Status 404 returned error can't find the container with id 6f3bf79b6459c185b7fe24bd3c2fa9ad1c9f4e96de8d4f0be677e3924fd90ac3 Sep 05 02:18:00.128204 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:00.127579 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306ce2c3_da47_4ecd_965a_5d2bda450872.slice/crio-315a964afaad3d0ddc359bf6c42f09a72fbf1a96e8deb2b062d6b3d6ef671148 WatchSource:0}: Error finding container 315a964afaad3d0ddc359bf6c42f09a72fbf1a96e8deb2b062d6b3d6ef671148: Status 404 returned error can't find the container with id 315a964afaad3d0ddc359bf6c42f09a72fbf1a96e8deb2b062d6b3d6ef671148 Sep 05 02:18:00.532069 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:00.531990 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:00.532069 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:00.532063 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:00.532263 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:00.532200 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:00.532296 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:00.532264 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs podName:257406f0-bf02-4fea-a10b-daf9750ddefc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:02.5322465 +0000 UTC m=+6.018105459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs") pod "network-metrics-daemon-r47jj" (UID: "257406f0-bf02-4fea-a10b-daf9750ddefc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:00.532414 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:00.532398 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:00.532460 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:00.532449 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret podName:d398890b-b9a3-4b54-abdd-d0f90f315b66 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:02.532436235 +0000 UTC m=+6.018295188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret") pod "global-pull-secret-syncer-74l8j" (UID: "d398890b-b9a3-4b54-abdd-d0f90f315b66") : object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:00.734172 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:00.734132 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:00.734373 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:00.734303 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 02:18:00.734373 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:00.734326 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 02:18:00.734373 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:00.734338 2567 projected.go:196] Error preparing data for projected volume kube-api-access-th29q for pod openshift-network-diagnostics/network-check-target-jslqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:00.734575 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:00.734395 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q podName:b5be9e27-b6e4-4477-9c67-2245faef62e9 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:02.734377858 +0000 UTC m=+6.220236815 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-th29q" (UniqueName: "kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q") pod "network-check-target-jslqd" (UID: "b5be9e27-b6e4-4477-9c67-2245faef62e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:00.815052 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:00.814966 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 02:18:01.018951 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.018017 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:01.018951 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:01.018161 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:01.018951 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.018578 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:01.018951 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:01.018675 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:01.018951 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.018742 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:01.018951 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:01.018812 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:01.059782 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.059742 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zgb7b" event={"ID":"fd04d4e5-d457-482a-b1ee-a2d606c18b81","Type":"ContainerStarted","Data":"95372790567a89f3517040a758ec735da70b2a68bb1fd65f8e26af04e979b44a"} Sep 05 02:18:01.064688 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.064655 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" event={"ID":"a4cb2d0a-0b28-4d06-a48f-ce729554a02e","Type":"ContainerStarted","Data":"99f456b5cf1a21674695a2f735b17c3bc7f42e9b03a1573f079c7428f20c117d"} Sep 05 02:18:01.079305 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.079230 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal" event={"ID":"07df7873c55aad720a5c1b311be3ff60","Type":"ContainerStarted","Data":"8f515ebe8a3dc60b215094253e6789d1458f211ed95194fc65677e384f4231fc"} Sep 05 02:18:01.082734 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.082608 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5bshb" event={"ID":"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb","Type":"ContainerStarted","Data":"e0933cb466f066a5c68904ec926e859d409a92ae178d2f2621a66c7d79fa2f76"} Sep 05 02:18:01.088149 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.088062 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p4p4n" event={"ID":"9e6dbc91-2b3f-40a1-9b30-525617d8c100","Type":"ContainerStarted","Data":"f9776adb30dcb721e02cbe7e99f15d57e09e410e6d7e68174dd3c3ce558cfb48"} Sep 05 02:18:01.096923 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.096776 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" event={"ID":"306ce2c3-da47-4ecd-965a-5d2bda450872","Type":"ContainerStarted","Data":"315a964afaad3d0ddc359bf6c42f09a72fbf1a96e8deb2b062d6b3d6ef671148"} Sep 05 02:18:01.104535 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.104457 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qx5mt" event={"ID":"687f9036-0a6d-4fb4-abef-d64801928cc1","Type":"ContainerStarted","Data":"6f3bf79b6459c185b7fe24bd3c2fa9ad1c9f4e96de8d4f0be677e3924fd90ac3"} Sep 05 02:18:01.127029 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.126991 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rrpll" event={"ID":"3132f740-a0ad-4ea5-be63-aaff35aa174f","Type":"ContainerStarted","Data":"e2b4cecdaaa2579c9b2be98273e71cf1df1cb0f3dc986c39f0b867337329b579"} Sep 05 02:18:01.137234 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.137195 2567 generic.go:356] "Generic (PLEG): container finished" podID="e20bfcd8fa7c3756f9a221829a74dd57" containerID="00c891fdb86b41f0da5e41562aac113d1dd9050bafb09957f1f53661c5e82b42" exitCode=0 Sep 05 02:18:01.137402 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.137293 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" event={"ID":"e20bfcd8fa7c3756f9a221829a74dd57","Type":"ContainerDied","Data":"00c891fdb86b41f0da5e41562aac113d1dd9050bafb09957f1f53661c5e82b42"} Sep 05 02:18:01.148741 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.148710 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" event={"ID":"4240570d-29f2-4cf0-b8f3-7e702ad70eaa","Type":"ContainerStarted","Data":"d3dfe839ea18bc5713fbaea7fcfa68a4462db5b87e324d30611374a6fb75da51"} Sep 05 02:18:01.153223 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.153194 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5q89w" event={"ID":"75cd35e8-1c2c-4bc6-a121-15adc834eb69","Type":"ContainerStarted","Data":"599900be781f7386c9cc21285f62f3073c8e9749a8368574ab90b48af5bd026b"} Sep 05 02:18:01.154374 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:01.154323 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-63.ec2.internal" podStartSLOduration=4.154304719 podStartE2EDuration="4.154304719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:01.095907385 +0000 UTC m=+4.581766361" watchObservedRunningTime="2026-09-05 02:18:01.154304719 +0000 UTC m=+4.640163694" Sep 05 02:18:02.174430 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:02.174394 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" event={"ID":"e20bfcd8fa7c3756f9a221829a74dd57","Type":"ContainerStarted","Data":"cd95a9b703c5d5434aefc9c78e8fb08e1522dd7542908c9606817ab6b60e11d2"} Sep 05 02:18:02.191881 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:02.191690 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-63.ec2.internal" podStartSLOduration=5.191667668 podStartE2EDuration="5.191667668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:02.190420119 +0000 UTC m=+5.676279096" watchObservedRunningTime="2026-09-05 02:18:02.191667668 +0000 UTC m=+5.677526644" Sep 05 02:18:02.548636 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:02.548515 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:02.548636 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:02.548633 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:02.548860 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:02.548768 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:02.548860 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:02.548824 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret podName:d398890b-b9a3-4b54-abdd-d0f90f315b66 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:06.548806052 +0000 UTC m=+10.034665012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret") pod "global-pull-secret-syncer-74l8j" (UID: "d398890b-b9a3-4b54-abdd-d0f90f315b66") : object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:02.549233 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:02.549212 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:02.549323 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:02.549263 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs podName:257406f0-bf02-4fea-a10b-daf9750ddefc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:06.549247473 +0000 UTC m=+10.035106431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs") pod "network-metrics-daemon-r47jj" (UID: "257406f0-bf02-4fea-a10b-daf9750ddefc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:02.750122 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:02.749994 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:02.750313 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:02.750189 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 02:18:02.750313 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:02.750208 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 02:18:02.750313 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:02.750218 2567 projected.go:196] Error preparing data for projected volume kube-api-access-th29q for pod openshift-network-diagnostics/network-check-target-jslqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:02.750313 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:02.750267 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q podName:b5be9e27-b6e4-4477-9c67-2245faef62e9 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:06.750250264 +0000 UTC m=+10.236109234 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-th29q" (UniqueName: "kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q") pod "network-check-target-jslqd" (UID: "b5be9e27-b6e4-4477-9c67-2245faef62e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:03.015084 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:03.015008 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:03.015244 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:03.015153 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:03.015540 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:03.015519 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:03.015645 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:03.015619 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:03.015721 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:03.015708 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:03.016157 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:03.015795 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:05.012883 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:05.012623 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:05.013328 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:05.013040 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:05.013328 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:05.013274 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:05.013451 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:05.013379 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:05.013692 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:05.013562 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:05.013692 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:05.013653 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:06.584530 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:06.583801 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:06.584530 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:06.583890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:06.584530 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:06.584038 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:06.584530 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:06.584088 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret podName:d398890b-b9a3-4b54-abdd-d0f90f315b66 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:14.58407511 +0000 UTC m=+18.069934063 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret") pod "global-pull-secret-syncer-74l8j" (UID: "d398890b-b9a3-4b54-abdd-d0f90f315b66") : object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:06.584530 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:06.584438 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:06.584530 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:06.584484 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs podName:257406f0-bf02-4fea-a10b-daf9750ddefc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:14.584469802 +0000 UTC m=+18.070328760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs") pod "network-metrics-daemon-r47jj" (UID: "257406f0-bf02-4fea-a10b-daf9750ddefc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:06.785602 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:06.785560 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:06.785798 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:06.785781 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 02:18:06.785870 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:06.785807 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 02:18:06.785870 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:06.785822 2567 projected.go:196] Error preparing data for projected volume kube-api-access-th29q for pod openshift-network-diagnostics/network-check-target-jslqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:06.785970 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:06.785890 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q podName:b5be9e27-b6e4-4477-9c67-2245faef62e9 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:14.785870391 +0000 UTC m=+18.271729350 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-th29q" (UniqueName: "kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q") pod "network-check-target-jslqd" (UID: "b5be9e27-b6e4-4477-9c67-2245faef62e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:07.014256 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:07.013486 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:07.014256 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:07.013632 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:07.014256 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:07.013977 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:07.014256 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:07.014067 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:07.014256 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:07.014112 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:07.014256 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:07.014175 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:09.012875 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:09.012842 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:09.013300 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:09.012970 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:09.013399 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:09.013381 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:09.013533 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:09.013511 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:09.013682 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:09.013653 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:09.013764 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:09.013740 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:11.014509 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:11.014465 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:11.014856 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:11.014533 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:11.014856 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:11.014626 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:11.014856 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:11.014707 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:11.014856 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:11.014758 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:11.014856 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:11.014810 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:13.012320 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:13.012255 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:13.012320 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:13.012277 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:13.012894 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:13.012383 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:13.012894 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:13.012572 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:13.012894 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:13.012687 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:13.012894 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:13.012763 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:14.642075 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:14.642040 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:14.642606 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:14.642114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:14.642606 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:14.642227 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:14.642606 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:14.642233 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:14.642606 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:14.642280 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs podName:257406f0-bf02-4fea-a10b-daf9750ddefc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:30.642261492 +0000 UTC m=+34.128120446 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs") pod "network-metrics-daemon-r47jj" (UID: "257406f0-bf02-4fea-a10b-daf9750ddefc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:14.642606 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:14.642297 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret podName:d398890b-b9a3-4b54-abdd-d0f90f315b66 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:30.642287978 +0000 UTC m=+34.128146937 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret") pod "global-pull-secret-syncer-74l8j" (UID: "d398890b-b9a3-4b54-abdd-d0f90f315b66") : object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:14.843894 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:14.843801 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:14.844055 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:14.844009 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 02:18:14.844055 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:14.844031 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 02:18:14.844055 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:14.844045 2567 projected.go:196] Error preparing data for projected volume kube-api-access-th29q for pod openshift-network-diagnostics/network-check-target-jslqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:14.844162 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:14.844096 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q podName:b5be9e27-b6e4-4477-9c67-2245faef62e9 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:30.844081808 +0000 UTC m=+34.329940761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-th29q" (UniqueName: "kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q") pod "network-check-target-jslqd" (UID: "b5be9e27-b6e4-4477-9c67-2245faef62e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:15.015770 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:15.015742 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:15.015970 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:15.015775 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:15.015970 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:15.015747 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:15.015970 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:15.015852 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:15.015970 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:15.015927 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:15.016182 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:15.015987 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:17.013329 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:17.013286 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:17.013864 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:17.013417 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:17.013864 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:17.013426 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:17.013864 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:17.013548 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:17.013864 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:17.013618 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:17.013864 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:17.013712 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:19.012524 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.012331 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:19.013130 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.012359 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:19.013130 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:19.012635 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:19.013130 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:19.012710 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:19.013130 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.012408 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:19.013130 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:19.012831 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:19.217474 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.217445 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" event={"ID":"4240570d-29f2-4cf0-b8f3-7e702ad70eaa","Type":"ContainerStarted","Data":"a364ba1e4954eecf3a84c5befff33b164e07337bc9a74ca001daecf5f54b3c4f"} Sep 05 02:18:19.219175 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.219152 2567 generic.go:356] "Generic (PLEG): container finished" podID="75cd35e8-1c2c-4bc6-a121-15adc834eb69" containerID="6e042db5fa23bde3b58ff11cf67db009d7392593a4df2c2529ea240482d37888" exitCode=0 Sep 05 02:18:19.219294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.219235 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5q89w" event={"ID":"75cd35e8-1c2c-4bc6-a121-15adc834eb69","Type":"ContainerDied","Data":"6e042db5fa23bde3b58ff11cf67db009d7392593a4df2c2529ea240482d37888"} Sep 05 02:18:19.221251 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.221223 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zgb7b" event={"ID":"fd04d4e5-d457-482a-b1ee-a2d606c18b81","Type":"ContainerStarted","Data":"a91c4aa89d0d2eb66aca2341776870f12f2a97d47f6dc45819ece1d2be7df01a"} Sep 05 02:18:19.223073 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.223051 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" event={"ID":"a4cb2d0a-0b28-4d06-a48f-ce729554a02e","Type":"ContainerStarted","Data":"86320fd7dfe639d307063be06ef8c116e24dfec35a62ed3a6e4c64f24a23e372"} Sep 05 02:18:19.224877 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.224754 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5bshb" event={"ID":"7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb","Type":"ContainerStarted","Data":"da1e16481562810b91d6c462eb3a9b3513d4665fcf6e8370516287daa90aabb8"} Sep 05 02:18:19.227694 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.227668 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p4p4n" event={"ID":"9e6dbc91-2b3f-40a1-9b30-525617d8c100","Type":"ContainerStarted","Data":"eccdc7b541fe91dba6b608f0c92ac7476c3acf01c1d970540a4c467ad45e3aa9"} Sep 05 02:18:19.233219 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.233192 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" event={"ID":"306ce2c3-da47-4ecd-965a-5d2bda450872","Type":"ContainerStarted","Data":"8debbcdbb428d7de574ea2092ba5f5023ebbe6ed4769f8cd71869b0a967f36fe"} Sep 05 02:18:19.233315 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.233274 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" event={"ID":"306ce2c3-da47-4ecd-965a-5d2bda450872","Type":"ContainerStarted","Data":"89bbc06d083da2b04021b55a167952424aa05874dea26baf0d57b601f450c5dc"} Sep 05 02:18:19.233357 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.233321 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" event={"ID":"306ce2c3-da47-4ecd-965a-5d2bda450872","Type":"ContainerStarted","Data":"0d764baf990c83e9e2104b3eaae4cd69dce04d8c2f82c4e285d5f5acf2bdf2c6"} Sep 05 02:18:19.233446 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.233367 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" event={"ID":"306ce2c3-da47-4ecd-965a-5d2bda450872","Type":"ContainerStarted","Data":"c1575dbe07b841ece38eec7762780af915ce9bcf4ed84b7122b55a68768f167e"} Sep 05 02:18:19.233446 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.233413 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" event={"ID":"306ce2c3-da47-4ecd-965a-5d2bda450872","Type":"ContainerStarted","Data":"9b6e44e9ba719becc5f89c4b1d02e0fc69dd86e2ea6d2a1678f73af9d390e3a7"} Sep 05 02:18:19.235902 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.235882 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qx5mt" event={"ID":"687f9036-0a6d-4fb4-abef-d64801928cc1","Type":"ContainerStarted","Data":"83fe2ed456d82355ab897db05b9063c6c81efc07730a5aa24eba04970c99631e"} Sep 05 02:18:19.263541 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.263469 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5bshb" podStartSLOduration=3.7609258199999998 podStartE2EDuration="22.263449042s" totalImagesPullingTime="18.502523222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:19.262720085 +0000 UTC m=+22.748579061" watchObservedRunningTime="2026-09-05 02:18:19.263449042 +0000 UTC m=+22.749308018" Sep 05 02:18:19.280881 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.280832 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-p4p4n" podStartSLOduration=4.032630401 podStartE2EDuration="22.280813385s" totalImagesPullingTime="18.248182984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:19.280400502 +0000 UTC m=+22.766259480" watchObservedRunningTime="2026-09-05 02:18:19.280813385 +0000 UTC m=+22.766672362" Sep 05 02:18:19.296675 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.296627 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5fc7x" podStartSLOduration=3.836785468 podStartE2EDuration="22.296607377s" totalImagesPullingTime="18.459821909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:19.296176166 +0000 UTC m=+22.782035141" watchObservedRunningTime="2026-09-05 02:18:19.296607377 +0000 UTC m=+22.782466356" Sep 05 02:18:19.310238 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.310200 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zgb7b" podStartSLOduration=7.97173263 podStartE2EDuration="22.310182721s" totalImagesPullingTime="14.338450091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:19.310144231 +0000 UTC m=+22.796003206" watchObservedRunningTime="2026-09-05 02:18:19.310182721 +0000 UTC m=+22.796041693" Sep 05 02:18:19.326626 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.326580 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qx5mt" podStartSLOduration=4.070253116 podStartE2EDuration="22.326566513s" totalImagesPullingTime="18.256313397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:19.326290574 +0000 UTC m=+22.812149549" watchObservedRunningTime="2026-09-05 02:18:19.326566513 +0000 UTC m=+22.812425487" Sep 05 02:18:19.724143 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.723983 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 05 02:18:19.981098 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.980992 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-05T02:18:19.724139751Z","UUID":"6ee98015-cfb1-4a06-bbb0-61977b3dab9f","Handler":null,"Name":"","Endpoint":""} Sep 05 02:18:19.982623 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.982603 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 05 02:18:19.982740 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:19.982636 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 05 02:18:20.243150 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:20.243042 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" event={"ID":"306ce2c3-da47-4ecd-965a-5d2bda450872","Type":"ContainerStarted","Data":"caeba84a2916251510cfbb21e2657c2661bb4e92d142b9920a570e8ba019c60d"} Sep 05 02:18:20.245821 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:20.245786 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rrpll" event={"ID":"3132f740-a0ad-4ea5-be63-aaff35aa174f","Type":"ContainerStarted","Data":"652989234e159b9a6fd3aa088a2558cb208d89a226e175d0a55c949741e7c7a3"} Sep 05 02:18:20.248376 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:20.248348 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" event={"ID":"4240570d-29f2-4cf0-b8f3-7e702ad70eaa","Type":"ContainerStarted","Data":"d7dbb0b81770f4b86c580aa7977d85f896adc2cd2d9ed4f4e7288a53a31ec6d6"} Sep 05 02:18:21.012098 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:21.012067 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:21.012279 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:21.012064 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:21.012279 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:21.012208 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:21.012279 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:21.012064 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:21.012414 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:21.012287 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:21.012414 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:21.012370 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:21.253595 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:21.253549 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" event={"ID":"4240570d-29f2-4cf0-b8f3-7e702ad70eaa","Type":"ContainerStarted","Data":"cb5c63c55fe55104acb92d3bbcddadfb01f31cc39ec95219bd7e754acbea1ccf"} Sep 05 02:18:21.273123 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:21.273069 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vb9c6" podStartSLOduration=4.060328207 podStartE2EDuration="24.273051963s" totalImagesPullingTime="20.212723756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:21.272763643 +0000 UTC m=+24.758622619" watchObservedRunningTime="2026-09-05 02:18:21.273051963 +0000 UTC m=+24.758910937" Sep 05 02:18:21.273574 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:21.273548 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rrpll" podStartSLOduration=5.818096625 podStartE2EDuration="24.273540473s" totalImagesPullingTime="18.455443848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:20.261552232 +0000 UTC m=+23.747411208" watchObservedRunningTime="2026-09-05 02:18:21.273540473 +0000 UTC m=+24.759399448" Sep 05 02:18:22.260051 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:22.260017 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" event={"ID":"306ce2c3-da47-4ecd-965a-5d2bda450872","Type":"ContainerStarted","Data":"ada92b487495d4398fc66a9aee437f0c4d4ebc9ccacecaee17b4ce64a3f863a3"} Sep 05 02:18:23.012062 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:23.012025 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:23.012299 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:23.012083 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:23.012299 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:23.012165 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:23.012299 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:23.012034 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:23.012299 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:23.012276 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:23.012478 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:23.012351 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:24.154934 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.154636 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:18:24.155442 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.155252 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:18:24.268028 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.267987 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" event={"ID":"306ce2c3-da47-4ecd-965a-5d2bda450872","Type":"ContainerStarted","Data":"fec7ae3115c0e9da52c788973b95310be4ab7046e03cffe38c007cb6f135ad83"} Sep 05 02:18:24.268297 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.268274 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:18:24.268375 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.268356 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:18:24.269941 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.269911 2567 generic.go:356] "Generic (PLEG): container finished" podID="75cd35e8-1c2c-4bc6-a121-15adc834eb69" containerID="ef8e4c9b463666416f9f12db1ecdcfa9bc42665d6194d56adc4081790929c6ad" exitCode=0 Sep 05 02:18:24.270049 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.269984 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5q89w" event={"ID":"75cd35e8-1c2c-4bc6-a121-15adc834eb69","Type":"ContainerDied","Data":"ef8e4c9b463666416f9f12db1ecdcfa9bc42665d6194d56adc4081790929c6ad"} Sep 05 02:18:24.270205 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.270188 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:18:24.270678 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.270665 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zgb7b" Sep 05 02:18:24.284529 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.284482 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:18:24.333831 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:24.333776 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" podStartSLOduration=8.692868845 podStartE2EDuration="27.333762164s" totalImagesPullingTime="18.640893319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:24.312618383 +0000 UTC m=+27.798477357" watchObservedRunningTime="2026-09-05 02:18:24.333762164 +0000 UTC m=+27.819621136" Sep 05 02:18:25.012714 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.012680 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:25.012857 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.012687 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:25.012857 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:25.012817 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:25.012956 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.012698 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:25.012956 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:25.012896 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:25.013023 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:25.012984 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:25.272168 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.272094 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:18:25.286448 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.286426 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:18:25.666762 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.666475 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-74l8j"] Sep 05 02:18:25.666928 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.666810 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:25.666928 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:25.666892 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:25.669329 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.669299 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r47jj"] Sep 05 02:18:25.669476 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.669419 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:25.669580 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:25.669556 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:25.669964 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.669944 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jslqd"] Sep 05 02:18:25.670078 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:25.670020 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:25.670136 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:25.670085 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:26.276136 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:26.276099 2567 generic.go:356] "Generic (PLEG): container finished" podID="75cd35e8-1c2c-4bc6-a121-15adc834eb69" containerID="398464a7d6ece8cb374a40c200e5642e712984d2d7dff03eda578e9b251c77c2" exitCode=0 Sep 05 02:18:26.276692 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:26.276180 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5q89w" event={"ID":"75cd35e8-1c2c-4bc6-a121-15adc834eb69","Type":"ContainerDied","Data":"398464a7d6ece8cb374a40c200e5642e712984d2d7dff03eda578e9b251c77c2"} Sep 05 02:18:27.013406 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:27.013383 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:27.013551 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:27.013458 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:27.013551 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:27.013531 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:27.013644 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:27.013622 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:27.280657 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:27.280628 2567 generic.go:356] "Generic (PLEG): container finished" podID="75cd35e8-1c2c-4bc6-a121-15adc834eb69" containerID="d8452fcb5ed45253960041dd6055b6c2ad55820d6c1577e731cc474593a8b6c7" exitCode=0 Sep 05 02:18:27.281014 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:27.280720 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5q89w" event={"ID":"75cd35e8-1c2c-4bc6-a121-15adc834eb69","Type":"ContainerDied","Data":"d8452fcb5ed45253960041dd6055b6c2ad55820d6c1577e731cc474593a8b6c7"} Sep 05 02:18:28.012318 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:28.012288 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:28.012475 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:28.012429 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:29.012130 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:29.012096 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:29.012621 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:29.012096 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:29.012621 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:29.012236 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:29.012621 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:29.012333 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:30.012455 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:30.012415 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:30.013020 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:30.012572 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:30.652468 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:30.652421 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:30.652770 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:30.652516 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:30.652770 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:30.652605 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:30.652770 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:30.652646 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:30.652770 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:30.652701 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret podName:d398890b-b9a3-4b54-abdd-d0f90f315b66 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:02.652683144 +0000 UTC m=+66.138542097 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret") pod "global-pull-secret-syncer-74l8j" (UID: "d398890b-b9a3-4b54-abdd-d0f90f315b66") : object "kube-system"/"original-pull-secret" not registered Sep 05 02:18:30.652770 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:30.652718 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs podName:257406f0-bf02-4fea-a10b-daf9750ddefc nodeName:}" failed. No retries permitted until 2026-09-05 02:19:02.652710728 +0000 UTC m=+66.138569681 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs") pod "network-metrics-daemon-r47jj" (UID: "257406f0-bf02-4fea-a10b-daf9750ddefc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 02:18:30.853967 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:30.853929 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:30.854153 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:30.854122 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 02:18:30.854153 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:30.854145 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 02:18:30.854153 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:30.854155 2567 projected.go:196] Error preparing data for projected volume kube-api-access-th29q for pod openshift-network-diagnostics/network-check-target-jslqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:30.854313 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:30.854211 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q podName:b5be9e27-b6e4-4477-9c67-2245faef62e9 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:02.854196455 +0000 UTC m=+66.340055413 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-th29q" (UniqueName: "kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q") pod "network-check-target-jslqd" (UID: "b5be9e27-b6e4-4477-9c67-2245faef62e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 02:18:31.012294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:31.012263 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:31.012473 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:31.012267 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:31.012473 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:31.012394 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-74l8j" podUID="d398890b-b9a3-4b54-abdd-d0f90f315b66" Sep 05 02:18:31.012855 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:31.012468 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jslqd" podUID="b5be9e27-b6e4-4477-9c67-2245faef62e9" Sep 05 02:18:32.012829 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.012799 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:32.013192 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:32.012903 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r47jj" podUID="257406f0-bf02-4fea-a10b-daf9750ddefc" Sep 05 02:18:32.292049 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.292019 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-63.ec2.internal" event="NodeReady" Sep 05 02:18:32.292191 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.292135 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 05 02:18:32.295894 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.295870 2567 generic.go:356] "Generic (PLEG): container finished" podID="75cd35e8-1c2c-4bc6-a121-15adc834eb69" containerID="b80a0b68d2c8def6d988d13e09fd192e17845366e9240af87cc6262f2ac48e52" exitCode=0 Sep 05 02:18:32.296011 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.295910 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5q89w" event={"ID":"75cd35e8-1c2c-4bc6-a121-15adc834eb69","Type":"ContainerDied","Data":"b80a0b68d2c8def6d988d13e09fd192e17845366e9240af87cc6262f2ac48e52"} Sep 05 02:18:32.336968 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.336940 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ghw92"] Sep 05 02:18:32.359400 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.359372 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wnvsv"] Sep 05 02:18:32.359547 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.359462 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.362183 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.362153 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 05 02:18:32.362530 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.362158 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 05 02:18:32.362530 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.362465 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cl7gt\"" Sep 05 02:18:32.383531 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.383506 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ghw92"] Sep 05 02:18:32.383621 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.383542 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wnvsv"] Sep 05 02:18:32.383621 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.383582 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wnvsv" Sep 05 02:18:32.386404 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.386344 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cd6qr\"" Sep 05 02:18:32.386404 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.386356 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 05 02:18:32.386404 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.386385 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 05 02:18:32.386708 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.386692 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 05 02:18:32.466029 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.466005 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-tmp-dir\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.466135 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.466041 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlfgp\" (UniqueName: \"kubernetes.io/projected/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-kube-api-access-xlfgp\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.466135 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.466117 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-config-volume\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.466224 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.466159 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a089884-83b9-4183-a0d5-3d1f2dc10ba3-cert\") pod \"ingress-canary-wnvsv\" (UID: \"4a089884-83b9-4183-a0d5-3d1f2dc10ba3\") " pod="openshift-ingress-canary/ingress-canary-wnvsv" Sep 05 02:18:32.466224 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.466174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.466224 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.466211 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtntf\" (UniqueName: \"kubernetes.io/projected/4a089884-83b9-4183-a0d5-3d1f2dc10ba3-kube-api-access-xtntf\") pod \"ingress-canary-wnvsv\" (UID: \"4a089884-83b9-4183-a0d5-3d1f2dc10ba3\") " pod="openshift-ingress-canary/ingress-canary-wnvsv" Sep 05 02:18:32.567325 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.567281 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a089884-83b9-4183-a0d5-3d1f2dc10ba3-cert\") pod \"ingress-canary-wnvsv\" (UID: \"4a089884-83b9-4183-a0d5-3d1f2dc10ba3\") " pod="openshift-ingress-canary/ingress-canary-wnvsv" Sep 05 02:18:32.567325 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.567325 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.567597 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.567362 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtntf\" (UniqueName: \"kubernetes.io/projected/4a089884-83b9-4183-a0d5-3d1f2dc10ba3-kube-api-access-xtntf\") pod \"ingress-canary-wnvsv\" (UID: \"4a089884-83b9-4183-a0d5-3d1f2dc10ba3\") " pod="openshift-ingress-canary/ingress-canary-wnvsv" Sep 05 02:18:32.567597 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.567390 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-tmp-dir\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.567597 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.567414 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlfgp\" (UniqueName: \"kubernetes.io/projected/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-kube-api-access-xlfgp\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.567597 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.567450 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-config-volume\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.567597 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:32.567450 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 02:18:32.567597 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:32.567568 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls podName:b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:33.067547968 +0000 UTC m=+36.553406941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls") pod "dns-default-ghw92" (UID: "b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc") : secret "dns-default-metrics-tls" not found Sep 05 02:18:32.567849 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.567809 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-tmp-dir\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.568071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.568048 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-config-volume\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.572122 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.571963 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a089884-83b9-4183-a0d5-3d1f2dc10ba3-cert\") pod \"ingress-canary-wnvsv\" (UID: \"4a089884-83b9-4183-a0d5-3d1f2dc10ba3\") " pod="openshift-ingress-canary/ingress-canary-wnvsv" Sep 05 02:18:32.576257 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.576231 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlfgp\" (UniqueName: \"kubernetes.io/projected/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-kube-api-access-xlfgp\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:32.576333 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.576265 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtntf\" (UniqueName: \"kubernetes.io/projected/4a089884-83b9-4183-a0d5-3d1f2dc10ba3-kube-api-access-xtntf\") pod \"ingress-canary-wnvsv\" (UID: \"4a089884-83b9-4183-a0d5-3d1f2dc10ba3\") " pod="openshift-ingress-canary/ingress-canary-wnvsv" Sep 05 02:18:32.694654 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:32.694576 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wnvsv" Sep 05 02:18:33.002454 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.002415 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wnvsv"] Sep 05 02:18:33.006556 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:33.006527 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a089884_83b9_4183_a0d5_3d1f2dc10ba3.slice/crio-4b1f88ee5a561fe054d7ff1788dc30a677d6f30c54b2d2587705b43c42daf9b2 WatchSource:0}: Error finding container 4b1f88ee5a561fe054d7ff1788dc30a677d6f30c54b2d2587705b43c42daf9b2: Status 404 returned error can't find the container with id 4b1f88ee5a561fe054d7ff1788dc30a677d6f30c54b2d2587705b43c42daf9b2 Sep 05 02:18:33.012995 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.012973 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:18:33.013406 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.013163 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:18:33.015522 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.015486 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 05 02:18:33.015691 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.015676 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hk4wx\"" Sep 05 02:18:33.015755 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.015678 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 05 02:18:33.015795 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.015679 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 05 02:18:33.071141 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.071112 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:33.071268 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:33.071251 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 02:18:33.071359 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:33.071310 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls podName:b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:34.071295053 +0000 UTC m=+37.557154005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls") pod "dns-default-ghw92" (UID: "b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc") : secret "dns-default-metrics-tls" not found Sep 05 02:18:33.301019 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.300939 2567 generic.go:356] "Generic (PLEG): container finished" podID="75cd35e8-1c2c-4bc6-a121-15adc834eb69" containerID="0c5961c022bf6cbedf276528e68b2d86dc6f46cc7a4fcc025f69b97040d5cfd3" exitCode=0 Sep 05 02:18:33.301019 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.300995 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5q89w" event={"ID":"75cd35e8-1c2c-4bc6-a121-15adc834eb69","Type":"ContainerDied","Data":"0c5961c022bf6cbedf276528e68b2d86dc6f46cc7a4fcc025f69b97040d5cfd3"} Sep 05 02:18:33.302122 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:33.302097 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wnvsv" event={"ID":"4a089884-83b9-4183-a0d5-3d1f2dc10ba3","Type":"ContainerStarted","Data":"4b1f88ee5a561fe054d7ff1788dc30a677d6f30c54b2d2587705b43c42daf9b2"} Sep 05 02:18:34.013089 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:34.012995 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:18:34.015810 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:34.015785 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 05 02:18:34.015810 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:34.015805 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4nzpg\"" Sep 05 02:18:34.080694 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:34.080224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:34.080694 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:34.080351 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 02:18:34.080694 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:34.080430 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls podName:b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:36.080408284 +0000 UTC m=+39.566267252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls") pod "dns-default-ghw92" (UID: "b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc") : secret "dns-default-metrics-tls" not found Sep 05 02:18:34.311189 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:34.311104 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5q89w" event={"ID":"75cd35e8-1c2c-4bc6-a121-15adc834eb69","Type":"ContainerStarted","Data":"b72f82e493136848e5d827058227f4aa2bb402c1a651e0f008d4ee049186a2f4"} Sep 05 02:18:34.336633 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:34.336572 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5q89w" podStartSLOduration=8.606874624 podStartE2EDuration="37.336480504s" totalImagesPullingTime="28.72960588s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:34.336144665 +0000 UTC m=+37.822003640" watchObservedRunningTime="2026-09-05 02:18:34.336480504 +0000 UTC m=+37.822339478" Sep 05 02:18:36.097112 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:36.096934 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:36.097528 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:36.097095 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 02:18:36.097528 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:36.097181 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls podName:b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:40.097166426 +0000 UTC m=+43.583025379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls") pod "dns-default-ghw92" (UID: "b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc") : secret "dns-default-metrics-tls" not found Sep 05 02:18:36.320415 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:36.320384 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wnvsv" event={"ID":"4a089884-83b9-4183-a0d5-3d1f2dc10ba3","Type":"ContainerStarted","Data":"6bc2992527cce5ada0afa75ddceecc5d556f292a590456774c26c94e1d3d10b6"} Sep 05 02:18:36.338449 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:36.338406 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wnvsv" podStartSLOduration=1.984997287 podStartE2EDuration="4.338392168s" totalImagesPullingTime="2.353394881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:36.337073077 +0000 UTC m=+39.822932051" watchObservedRunningTime="2026-09-05 02:18:36.338392168 +0000 UTC m=+39.824251142" Sep 05 02:18:40.125338 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:40.125295 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:40.125769 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:40.125476 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 02:18:40.125769 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:40.125563 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls podName:b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc nodeName:}" failed. No retries permitted until 2026-09-05 02:18:48.125547191 +0000 UTC m=+51.611406144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls") pod "dns-default-ghw92" (UID: "b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc") : secret "dns-default-metrics-tls" not found Sep 05 02:18:48.178181 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:48.178132 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:18:48.178665 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:48.178285 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 02:18:48.178665 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:48.178347 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls podName:b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc nodeName:}" failed. No retries permitted until 2026-09-05 02:19:04.178332039 +0000 UTC m=+67.664190992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls") pod "dns-default-ghw92" (UID: "b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc") : secret "dns-default-metrics-tls" not found Sep 05 02:18:52.740598 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.740562 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z"] Sep 05 02:18:52.742459 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.742438 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:52.744948 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.744914 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 05 02:18:52.745375 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.745357 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-94rzh\"" Sep 05 02:18:52.745414 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.745360 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 05 02:18:52.745972 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.745957 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 05 02:18:52.746046 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.745957 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 05 02:18:52.749615 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.749597 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc"] Sep 05 02:18:52.751168 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.751151 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f"] Sep 05 02:18:52.751287 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.751272 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:18:52.754084 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.754065 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-92bgx"] Sep 05 02:18:52.755009 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.754755 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:52.755009 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.754959 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 05 02:18:52.755125 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.754993 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5n7ln\"" Sep 05 02:18:52.755125 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.755100 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 05 02:18:52.755871 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.755855 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 05 02:18:52.756832 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.756814 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:52.757362 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.757346 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 05 02:18:52.758065 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.758045 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 05 02:18:52.759539 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.759512 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hfhzw\"" Sep 05 02:18:52.759651 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.759604 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 05 02:18:52.759872 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.759855 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 05 02:18:52.759956 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.759910 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 05 02:18:52.760300 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.760277 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 05 02:18:52.760474 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.760309 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sj9k6\"" Sep 05 02:18:52.760561 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.760533 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 05 02:18:52.760617 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.760571 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 05 02:18:52.760866 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.760796 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z"] Sep 05 02:18:52.766286 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.766267 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc"] Sep 05 02:18:52.766988 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.766970 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-92bgx"] Sep 05 02:18:52.767294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.767276 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 05 02:18:52.779094 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.779075 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f"] Sep 05 02:18:52.862830 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.862796 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-cfg6p"] Sep 05 02:18:52.864538 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.864518 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm"] Sep 05 02:18:52.864661 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.864633 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:52.866104 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.866087 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm" Sep 05 02:18:52.869454 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.869434 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-n85pc\"" Sep 05 02:18:52.869588 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.869488 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 05 02:18:52.870031 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.870012 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 05 02:18:52.870160 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.870145 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 05 02:18:52.870212 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.870146 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9v55x\"" Sep 05 02:18:52.870480 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.870463 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 05 02:18:52.870876 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.870860 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 05 02:18:52.872484 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.872466 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 05 02:18:52.876997 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.876977 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 05 02:18:52.881382 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.881362 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm"] Sep 05 02:18:52.882082 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.882062 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-cfg6p"] Sep 05 02:18:52.912031 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912008 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b4e3d1f-b577-414a-a0e4-882b83ae9d45-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rr89z\" (UID: \"0b4e3d1f-b577-414a-a0e4-882b83ae9d45\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:52.912152 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912039 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzhf6\" (UniqueName: \"kubernetes.io/projected/0b4e3d1f-b577-414a-a0e4-882b83ae9d45-kube-api-access-xzhf6\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rr89z\" (UID: \"0b4e3d1f-b577-414a-a0e4-882b83ae9d45\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:52.912152 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912057 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b525226-203f-4f6f-91ed-93f07eb5653f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:52.912152 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912074 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rzp6\" (UniqueName: \"kubernetes.io/projected/9facd67b-62a7-4e19-b399-19b68d0d97a4-kube-api-access-4rzp6\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:52.912152 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912089 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b4e3d1f-b577-414a-a0e4-882b83ae9d45-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rr89z\" (UID: \"0b4e3d1f-b577-414a-a0e4-882b83ae9d45\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:52.912152 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912107 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b525226-203f-4f6f-91ed-93f07eb5653f-serving-cert\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:52.912310 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b525226-203f-4f6f-91ed-93f07eb5653f-tmp\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:52.912310 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912200 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b525226-203f-4f6f-91ed-93f07eb5653f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:52.912310 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912219 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:52.912310 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912235 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:18:52.912310 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912251 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq2rs\" (UniqueName: \"kubernetes.io/projected/cb2ec8db-6ba7-4221-b886-3a679682581d-kube-api-access-nq2rs\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:18:52.912310 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912275 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6c4s\" (UniqueName: \"kubernetes.io/projected/0b525226-203f-4f6f-91ed-93f07eb5653f-kube-api-access-n6c4s\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:52.912310 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912292 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9facd67b-62a7-4e19-b399-19b68d0d97a4-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:52.912310 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:52.912308 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b525226-203f-4f6f-91ed-93f07eb5653f-snapshots\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.012753 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.012665 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b525226-203f-4f6f-91ed-93f07eb5653f-tmp\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.012753 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.012701 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b525226-203f-4f6f-91ed-93f07eb5653f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.012753 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.012730 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kcb9\" (UniqueName: \"kubernetes.io/projected/af6ef192-9048-4fb7-9b65-b2de6f8a7048-kube-api-access-9kcb9\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.012992 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.012760 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:53.012992 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.012788 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:18:53.012992 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.012812 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq2rs\" (UniqueName: \"kubernetes.io/projected/cb2ec8db-6ba7-4221-b886-3a679682581d-kube-api-access-nq2rs\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:18:53.012992 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:53.012873 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 02:18:53.012992 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:53.012909 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 05 02:18:53.012992 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.012949 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6c4s\" (UniqueName: \"kubernetes.io/projected/0b525226-203f-4f6f-91ed-93f07eb5653f-kube-api-access-n6c4s\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.012992 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:53.012958 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls podName:9facd67b-62a7-4e19-b399-19b68d0d97a4 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:53.51293294 +0000 UTC m=+56.998791913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lfv7f" (UID: "9facd67b-62a7-4e19-b399-19b68d0d97a4") : secret "cluster-monitoring-operator-tls" not found Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:53.013000 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls podName:cb2ec8db-6ba7-4221-b886-3a679682581d nodeName:}" failed. No retries permitted until 2026-09-05 02:18:53.512980355 +0000 UTC m=+56.998839316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wn8qc" (UID: "cb2ec8db-6ba7-4221-b886-3a679682581d") : secret "samples-operator-tls" not found Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013034 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9facd67b-62a7-4e19-b399-19b68d0d97a4-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013062 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b525226-203f-4f6f-91ed-93f07eb5653f-snapshots\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013096 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j5kf\" (UniqueName: \"kubernetes.io/projected/8e504a92-8f83-4d1b-91b6-c3ef4babbaaf-kube-api-access-2j5kf\") pod \"volume-data-source-validator-695c895bd6-sj2lm\" (UID: \"8e504a92-8f83-4d1b-91b6-c3ef4babbaaf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm" Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013104 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b525226-203f-4f6f-91ed-93f07eb5653f-tmp\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013163 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af6ef192-9048-4fb7-9b65-b2de6f8a7048-serving-cert\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013195 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af6ef192-9048-4fb7-9b65-b2de6f8a7048-trusted-ca\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013267 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b4e3d1f-b577-414a-a0e4-882b83ae9d45-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rr89z\" (UID: \"0b4e3d1f-b577-414a-a0e4-882b83ae9d45\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013301 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzhf6\" (UniqueName: \"kubernetes.io/projected/0b4e3d1f-b577-414a-a0e4-882b83ae9d45-kube-api-access-xzhf6\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rr89z\" (UID: \"0b4e3d1f-b577-414a-a0e4-882b83ae9d45\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:53.013336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013325 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b525226-203f-4f6f-91ed-93f07eb5653f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.013842 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013353 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rzp6\" (UniqueName: \"kubernetes.io/projected/9facd67b-62a7-4e19-b399-19b68d0d97a4-kube-api-access-4rzp6\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:53.013842 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013377 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b4e3d1f-b577-414a-a0e4-882b83ae9d45-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rr89z\" (UID: \"0b4e3d1f-b577-414a-a0e4-882b83ae9d45\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:53.013842 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013400 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b525226-203f-4f6f-91ed-93f07eb5653f-serving-cert\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.013842 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013398 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b525226-203f-4f6f-91ed-93f07eb5653f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.013842 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013429 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af6ef192-9048-4fb7-9b65-b2de6f8a7048-config\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.013842 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013644 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b4e3d1f-b577-414a-a0e4-882b83ae9d45-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rr89z\" (UID: \"0b4e3d1f-b577-414a-a0e4-882b83ae9d45\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:53.013842 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013697 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b525226-203f-4f6f-91ed-93f07eb5653f-snapshots\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.013842 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.013787 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9facd67b-62a7-4e19-b399-19b68d0d97a4-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:53.014372 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.014279 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b525226-203f-4f6f-91ed-93f07eb5653f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.016824 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.016802 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b525226-203f-4f6f-91ed-93f07eb5653f-serving-cert\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.016824 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.016813 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b4e3d1f-b577-414a-a0e4-882b83ae9d45-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rr89z\" (UID: \"0b4e3d1f-b577-414a-a0e4-882b83ae9d45\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:53.026915 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.026886 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rzp6\" (UniqueName: \"kubernetes.io/projected/9facd67b-62a7-4e19-b399-19b68d0d97a4-kube-api-access-4rzp6\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:53.027089 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.027067 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzhf6\" (UniqueName: \"kubernetes.io/projected/0b4e3d1f-b577-414a-a0e4-882b83ae9d45-kube-api-access-xzhf6\") pod \"kube-storage-version-migrator-operator-8d7dd4675-rr89z\" (UID: \"0b4e3d1f-b577-414a-a0e4-882b83ae9d45\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:53.027147 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.027097 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq2rs\" (UniqueName: \"kubernetes.io/projected/cb2ec8db-6ba7-4221-b886-3a679682581d-kube-api-access-nq2rs\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:18:53.027147 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.027108 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6c4s\" (UniqueName: \"kubernetes.io/projected/0b525226-203f-4f6f-91ed-93f07eb5653f-kube-api-access-n6c4s\") pod \"insights-operator-76bcd6ccf5-92bgx\" (UID: \"0b525226-203f-4f6f-91ed-93f07eb5653f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.051902 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.051879 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" Sep 05 02:18:53.076406 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.076378 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" Sep 05 02:18:53.114378 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.114024 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af6ef192-9048-4fb7-9b65-b2de6f8a7048-config\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.114378 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.114085 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kcb9\" (UniqueName: \"kubernetes.io/projected/af6ef192-9048-4fb7-9b65-b2de6f8a7048-kube-api-access-9kcb9\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.114378 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.114156 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j5kf\" (UniqueName: \"kubernetes.io/projected/8e504a92-8f83-4d1b-91b6-c3ef4babbaaf-kube-api-access-2j5kf\") pod \"volume-data-source-validator-695c895bd6-sj2lm\" (UID: \"8e504a92-8f83-4d1b-91b6-c3ef4babbaaf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm" Sep 05 02:18:53.114378 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.114212 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af6ef192-9048-4fb7-9b65-b2de6f8a7048-serving-cert\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.114378 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.114243 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af6ef192-9048-4fb7-9b65-b2de6f8a7048-trusted-ca\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.116457 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.116378 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af6ef192-9048-4fb7-9b65-b2de6f8a7048-config\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.116766 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.116718 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af6ef192-9048-4fb7-9b65-b2de6f8a7048-trusted-ca\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.119505 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.119415 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af6ef192-9048-4fb7-9b65-b2de6f8a7048-serving-cert\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.123451 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.123428 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j5kf\" (UniqueName: \"kubernetes.io/projected/8e504a92-8f83-4d1b-91b6-c3ef4babbaaf-kube-api-access-2j5kf\") pod \"volume-data-source-validator-695c895bd6-sj2lm\" (UID: \"8e504a92-8f83-4d1b-91b6-c3ef4babbaaf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm" Sep 05 02:18:53.123604 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.123586 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kcb9\" (UniqueName: \"kubernetes.io/projected/af6ef192-9048-4fb7-9b65-b2de6f8a7048-kube-api-access-9kcb9\") pod \"console-operator-85b454874b-cfg6p\" (UID: \"af6ef192-9048-4fb7-9b65-b2de6f8a7048\") " pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.174529 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.174477 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:18:53.179861 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.179840 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm" Sep 05 02:18:53.319550 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.319474 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z"] Sep 05 02:18:53.322829 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:53.322804 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b4e3d1f_b577_414a_a0e4_882b83ae9d45.slice/crio-aabe3b1739854acd1efd77bd133dfbc386450d8c972e89c927120073c625295d WatchSource:0}: Error finding container aabe3b1739854acd1efd77bd133dfbc386450d8c972e89c927120073c625295d: Status 404 returned error can't find the container with id aabe3b1739854acd1efd77bd133dfbc386450d8c972e89c927120073c625295d Sep 05 02:18:53.348653 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.348621 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-92bgx"] Sep 05 02:18:53.352714 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:53.352688 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b525226_203f_4f6f_91ed_93f07eb5653f.slice/crio-ebf1f4540d804e67c5b525ba647401779206ef51198988c12188740b3e1fae8a WatchSource:0}: Error finding container ebf1f4540d804e67c5b525ba647401779206ef51198988c12188740b3e1fae8a: Status 404 returned error can't find the container with id ebf1f4540d804e67c5b525ba647401779206ef51198988c12188740b3e1fae8a Sep 05 02:18:53.361386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.361362 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" event={"ID":"0b525226-203f-4f6f-91ed-93f07eb5653f","Type":"ContainerStarted","Data":"ebf1f4540d804e67c5b525ba647401779206ef51198988c12188740b3e1fae8a"} Sep 05 02:18:53.362449 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.362425 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" event={"ID":"0b4e3d1f-b577-414a-a0e4-882b83ae9d45","Type":"ContainerStarted","Data":"aabe3b1739854acd1efd77bd133dfbc386450d8c972e89c927120073c625295d"} Sep 05 02:18:53.449309 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.449281 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-cfg6p"] Sep 05 02:18:53.451257 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.451237 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm"] Sep 05 02:18:53.452796 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:53.452774 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6ef192_9048_4fb7_9b65_b2de6f8a7048.slice/crio-fa7d7affae0a4713d0a6b6b6f9a72df7f91a54af7b69c499d0a165bebb330b9c WatchSource:0}: Error finding container fa7d7affae0a4713d0a6b6b6f9a72df7f91a54af7b69c499d0a165bebb330b9c: Status 404 returned error can't find the container with id fa7d7affae0a4713d0a6b6b6f9a72df7f91a54af7b69c499d0a165bebb330b9c Sep 05 02:18:53.454110 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:53.454078 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e504a92_8f83_4d1b_91b6_c3ef4babbaaf.slice/crio-557e2114db8cddff837258b521fa36b8aa68c3d377e17665a3d687ce2351b486 WatchSource:0}: Error finding container 557e2114db8cddff837258b521fa36b8aa68c3d377e17665a3d687ce2351b486: Status 404 returned error can't find the container with id 557e2114db8cddff837258b521fa36b8aa68c3d377e17665a3d687ce2351b486 Sep 05 02:18:53.517992 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.517960 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:53.517992 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:53.517992 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:18:53.518185 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:53.518109 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 05 02:18:53.518185 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:53.518109 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 02:18:53.518185 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:53.518173 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls podName:cb2ec8db-6ba7-4221-b886-3a679682581d nodeName:}" failed. No retries permitted until 2026-09-05 02:18:54.518156042 +0000 UTC m=+58.004015012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wn8qc" (UID: "cb2ec8db-6ba7-4221-b886-3a679682581d") : secret "samples-operator-tls" not found Sep 05 02:18:53.518295 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:53.518237 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls podName:9facd67b-62a7-4e19-b399-19b68d0d97a4 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:54.518224866 +0000 UTC m=+58.004083821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lfv7f" (UID: "9facd67b-62a7-4e19-b399-19b68d0d97a4") : secret "cluster-monitoring-operator-tls" not found Sep 05 02:18:54.369722 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:54.369680 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm" event={"ID":"8e504a92-8f83-4d1b-91b6-c3ef4babbaaf","Type":"ContainerStarted","Data":"557e2114db8cddff837258b521fa36b8aa68c3d377e17665a3d687ce2351b486"} Sep 05 02:18:54.371984 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:54.371949 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" event={"ID":"af6ef192-9048-4fb7-9b65-b2de6f8a7048","Type":"ContainerStarted","Data":"fa7d7affae0a4713d0a6b6b6f9a72df7f91a54af7b69c499d0a165bebb330b9c"} Sep 05 02:18:54.526670 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:54.526264 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:54.526670 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:54.526323 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:18:54.526670 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:54.526405 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 02:18:54.526670 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:54.526482 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls podName:9facd67b-62a7-4e19-b399-19b68d0d97a4 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:56.526461574 +0000 UTC m=+60.012320540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lfv7f" (UID: "9facd67b-62a7-4e19-b399-19b68d0d97a4") : secret "cluster-monitoring-operator-tls" not found Sep 05 02:18:54.526670 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:54.526548 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 05 02:18:54.526670 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:54.526607 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls podName:cb2ec8db-6ba7-4221-b886-3a679682581d nodeName:}" failed. No retries permitted until 2026-09-05 02:18:56.526591552 +0000 UTC m=+60.012450524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wn8qc" (UID: "cb2ec8db-6ba7-4221-b886-3a679682581d") : secret "samples-operator-tls" not found Sep 05 02:18:56.251286 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.251245 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-7f8gm"] Sep 05 02:18:56.253432 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.253407 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-7f8gm" Sep 05 02:18:56.255973 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.255951 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-l5fcv\"" Sep 05 02:18:56.259847 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.259820 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-7f8gm"] Sep 05 02:18:56.343201 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.343162 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pppv\" (UniqueName: \"kubernetes.io/projected/00e747c8-fb32-431b-ab52-98b738cca93e-kube-api-access-9pppv\") pod \"network-check-source-685456655f-7f8gm\" (UID: \"00e747c8-fb32-431b-ab52-98b738cca93e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-7f8gm" Sep 05 02:18:56.444843 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.444801 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pppv\" (UniqueName: \"kubernetes.io/projected/00e747c8-fb32-431b-ab52-98b738cca93e-kube-api-access-9pppv\") pod \"network-check-source-685456655f-7f8gm\" (UID: \"00e747c8-fb32-431b-ab52-98b738cca93e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-7f8gm" Sep 05 02:18:56.459446 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.459414 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pppv\" (UniqueName: \"kubernetes.io/projected/00e747c8-fb32-431b-ab52-98b738cca93e-kube-api-access-9pppv\") pod \"network-check-source-685456655f-7f8gm\" (UID: \"00e747c8-fb32-431b-ab52-98b738cca93e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-7f8gm" Sep 05 02:18:56.546124 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.546096 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:18:56.546263 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.546133 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:18:56.546263 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:56.546205 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 02:18:56.546369 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:56.546273 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 05 02:18:56.546369 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:56.546288 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls podName:9facd67b-62a7-4e19-b399-19b68d0d97a4 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:00.546270459 +0000 UTC m=+64.032129432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lfv7f" (UID: "9facd67b-62a7-4e19-b399-19b68d0d97a4") : secret "cluster-monitoring-operator-tls" not found Sep 05 02:18:56.546369 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:56.546321 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls podName:cb2ec8db-6ba7-4221-b886-3a679682581d nodeName:}" failed. No retries permitted until 2026-09-05 02:19:00.546308058 +0000 UTC m=+64.032167013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wn8qc" (UID: "cb2ec8db-6ba7-4221-b886-3a679682581d") : secret "samples-operator-tls" not found Sep 05 02:18:56.563561 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.563537 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-7f8gm" Sep 05 02:18:56.846629 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:56.846557 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-7f8gm"] Sep 05 02:18:56.849452 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:18:56.849429 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e747c8_fb32_431b_ab52_98b738cca93e.slice/crio-944417835dc710e5141412e11b453946542a7c2c831dc5444c591bc59aae4f07 WatchSource:0}: Error finding container 944417835dc710e5141412e11b453946542a7c2c831dc5444c591bc59aae4f07: Status 404 returned error can't find the container with id 944417835dc710e5141412e11b453946542a7c2c831dc5444c591bc59aae4f07 Sep 05 02:18:57.292862 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.292833 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8cc78" Sep 05 02:18:57.381690 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.381648 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" event={"ID":"0b525226-203f-4f6f-91ed-93f07eb5653f","Type":"ContainerStarted","Data":"313762c7d3ff7431ce7bf97f353a8f49c1e623954190bd1a169d0bd34eaa7bf4"} Sep 05 02:18:57.383765 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.383742 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/0.log" Sep 05 02:18:57.383903 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.383788 2567 generic.go:356] "Generic (PLEG): container finished" podID="af6ef192-9048-4fb7-9b65-b2de6f8a7048" containerID="de5852ad34ab20a5193ede13f66c405203e398a2b0afc8b3de7392036a41b860" exitCode=255 Sep 05 02:18:57.383903 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.383885 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" event={"ID":"af6ef192-9048-4fb7-9b65-b2de6f8a7048","Type":"ContainerDied","Data":"de5852ad34ab20a5193ede13f66c405203e398a2b0afc8b3de7392036a41b860"} Sep 05 02:18:57.384340 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.384038 2567 scope.go:117] "RemoveContainer" containerID="de5852ad34ab20a5193ede13f66c405203e398a2b0afc8b3de7392036a41b860" Sep 05 02:18:57.386253 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.386221 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" event={"ID":"0b4e3d1f-b577-414a-a0e4-882b83ae9d45","Type":"ContainerStarted","Data":"421bbe3b4c99100ae01752c4af12976829711e92941358854c0636a665f93c69"} Sep 05 02:18:57.387968 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.387946 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-7f8gm" event={"ID":"00e747c8-fb32-431b-ab52-98b738cca93e","Type":"ContainerStarted","Data":"944417835dc710e5141412e11b453946542a7c2c831dc5444c591bc59aae4f07"} Sep 05 02:18:57.389916 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.389895 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm" event={"ID":"8e504a92-8f83-4d1b-91b6-c3ef4babbaaf","Type":"ContainerStarted","Data":"afc69409f1e731e0ee0a4bf0a52f9b9a2ca4b20212c6b9211e14025cb70ee6c3"} Sep 05 02:18:57.399793 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.399736 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" podStartSLOduration=2.206505686 podStartE2EDuration="5.399718674s" totalImagesPullingTime="3.193212988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:57.399404165 +0000 UTC m=+60.885263141" watchObservedRunningTime="2026-09-05 02:18:57.399718674 +0000 UTC m=+60.885577650" Sep 05 02:18:57.452879 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.452345 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" podStartSLOduration=2.221619719 podStartE2EDuration="5.452323073s" totalImagesPullingTime="3.230703354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:57.423729925 +0000 UTC m=+60.909588903" watchObservedRunningTime="2026-09-05 02:18:57.452323073 +0000 UTC m=+60.938182049" Sep 05 02:18:57.474034 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.473982 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-sj2lm" podStartSLOduration=2.381584064 podStartE2EDuration="5.473962222s" totalImagesPullingTime="3.092378158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:18:57.472656138 +0000 UTC m=+60.958515114" watchObservedRunningTime="2026-09-05 02:18:57.473962222 +0000 UTC m=+60.959821224" Sep 05 02:18:57.642756 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.642673 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hp58v"] Sep 05 02:18:57.645260 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.645237 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.660708 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.660604 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 05 02:18:57.660866 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.660747 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 05 02:18:57.661116 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.661095 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7x6sm\"" Sep 05 02:18:57.695201 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.695173 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hp58v"] Sep 05 02:18:57.757716 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.757679 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/611d2010-264b-444d-8bf2-f856af766d89-data-volume\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.757904 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.757722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.757904 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.757866 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcm5z\" (UniqueName: \"kubernetes.io/projected/611d2010-264b-444d-8bf2-f856af766d89-kube-api-access-fcm5z\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.758044 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.757986 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/611d2010-264b-444d-8bf2-f856af766d89-crio-socket\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.758044 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.758021 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.859237 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.859201 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/611d2010-264b-444d-8bf2-f856af766d89-data-volume\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.859237 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.859240 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.859458 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:57.859346 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 02:18:57.859458 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.859401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcm5z\" (UniqueName: \"kubernetes.io/projected/611d2010-264b-444d-8bf2-f856af766d89-kube-api-access-fcm5z\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.859458 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:57.859416 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:58.3593963 +0000 UTC m=+61.845255269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : configmap "kube-rbac-proxy" not found Sep 05 02:18:57.859657 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.859489 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/611d2010-264b-444d-8bf2-f856af766d89-crio-socket\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.859657 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.859546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.859657 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.859563 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/611d2010-264b-444d-8bf2-f856af766d89-crio-socket\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.859657 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:57.859643 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 02:18:57.859849 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:57.859686 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:58.359673778 +0000 UTC m=+61.845532735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : secret "insights-runtime-extractor-tls" not found Sep 05 02:18:57.859849 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.859688 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/611d2010-264b-444d-8bf2-f856af766d89-data-volume\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:57.869785 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:57.869760 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcm5z\" (UniqueName: \"kubernetes.io/projected/611d2010-264b-444d-8bf2-f856af766d89-kube-api-access-fcm5z\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:58.363836 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:58.363803 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:58.364266 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:58.363867 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:58.364266 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:58.363989 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 02:18:58.364266 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:58.363999 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 02:18:58.364266 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:58.364070 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:59.364049353 +0000 UTC m=+62.849908312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : secret "insights-runtime-extractor-tls" not found Sep 05 02:18:58.364266 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:58.364090 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:18:59.364081154 +0000 UTC m=+62.849940112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : configmap "kube-rbac-proxy" not found Sep 05 02:18:58.400397 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:58.400303 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/1.log" Sep 05 02:18:58.400839 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:58.400822 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/0.log" Sep 05 02:18:58.400959 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:58.400860 2567 generic.go:356] "Generic (PLEG): container finished" podID="af6ef192-9048-4fb7-9b65-b2de6f8a7048" containerID="5937dee3ed4552bfeaf46b5bfef1381d5f1ddfb6f2c1374992dcb977a743d774" exitCode=255 Sep 05 02:18:58.401012 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:58.400966 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" event={"ID":"af6ef192-9048-4fb7-9b65-b2de6f8a7048","Type":"ContainerDied","Data":"5937dee3ed4552bfeaf46b5bfef1381d5f1ddfb6f2c1374992dcb977a743d774"} Sep 05 02:18:58.401064 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:58.401056 2567 scope.go:117] "RemoveContainer" containerID="de5852ad34ab20a5193ede13f66c405203e398a2b0afc8b3de7392036a41b860" Sep 05 02:18:58.401213 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:58.401187 2567 scope.go:117] "RemoveContainer" containerID="5937dee3ed4552bfeaf46b5bfef1381d5f1ddfb6f2c1374992dcb977a743d774" Sep 05 02:18:58.401453 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:58.401430 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-cfg6p_openshift-console-operator(af6ef192-9048-4fb7-9b65-b2de6f8a7048)\"" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" podUID="af6ef192-9048-4fb7-9b65-b2de6f8a7048" Sep 05 02:18:59.373872 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:59.373829 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:59.374378 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:59.373921 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:18:59.374378 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:59.374009 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 02:18:59.374378 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:59.374025 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 02:18:59.374378 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:59.374088 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:01.374067924 +0000 UTC m=+64.859926879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : configmap "kube-rbac-proxy" not found Sep 05 02:18:59.374378 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:59.374106 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:01.374096859 +0000 UTC m=+64.859955816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : secret "insights-runtime-extractor-tls" not found Sep 05 02:18:59.405548 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:59.405520 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/1.log" Sep 05 02:18:59.405952 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:59.405930 2567 scope.go:117] "RemoveContainer" containerID="5937dee3ed4552bfeaf46b5bfef1381d5f1ddfb6f2c1374992dcb977a743d774" Sep 05 02:18:59.406134 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:18:59.406114 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-cfg6p_openshift-console-operator(af6ef192-9048-4fb7-9b65-b2de6f8a7048)\"" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" podUID="af6ef192-9048-4fb7-9b65-b2de6f8a7048" Sep 05 02:18:59.770622 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:18:59.770594 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p4p4n_9e6dbc91-2b3f-40a1-9b30-525617d8c100/dns-node-resolver/0.log" Sep 05 02:19:00.370700 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:00.370672 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qx5mt_687f9036-0a6d-4fb4-abef-d64801928cc1/node-ca/0.log" Sep 05 02:19:00.410619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:00.410586 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-7f8gm" event={"ID":"00e747c8-fb32-431b-ab52-98b738cca93e","Type":"ContainerStarted","Data":"c730a26e29a54e15097f8722cb78b0906c3daaac61f600d22742c14d6b288770"} Sep 05 02:19:00.428081 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:00.428031 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-7f8gm" podStartSLOduration=1.6037794939999999 podStartE2EDuration="4.428015384s" totalImagesPullingTime="2.82423589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:00.427730937 +0000 UTC m=+63.913589911" watchObservedRunningTime="2026-09-05 02:19:00.428015384 +0000 UTC m=+63.913874358" Sep 05 02:19:00.583748 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:00.583710 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:19:00.583914 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:00.583870 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 02:19:00.583977 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:00.583959 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:19:00.584018 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:00.583988 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls podName:9facd67b-62a7-4e19-b399-19b68d0d97a4 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:08.583966959 +0000 UTC m=+72.069825926 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lfv7f" (UID: "9facd67b-62a7-4e19-b399-19b68d0d97a4") : secret "cluster-monitoring-operator-tls" not found Sep 05 02:19:00.584069 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:00.584041 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 05 02:19:00.584103 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:00.584083 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls podName:cb2ec8db-6ba7-4221-b886-3a679682581d nodeName:}" failed. No retries permitted until 2026-09-05 02:19:08.584070668 +0000 UTC m=+72.069929635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wn8qc" (UID: "cb2ec8db-6ba7-4221-b886-3a679682581d") : secret "samples-operator-tls" not found Sep 05 02:19:01.370164 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:01.370131 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wnvsv_4a089884-83b9-4183-a0d5-3d1f2dc10ba3/serve-healthcheck-canary/0.log" Sep 05 02:19:01.392743 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:01.392709 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:19:01.392913 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:01.392788 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:19:01.392913 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:01.392868 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 02:19:01.393027 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:01.392867 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 02:19:01.393027 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:01.392937 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:05.392921908 +0000 UTC m=+68.878780862 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : secret "insights-runtime-extractor-tls" not found Sep 05 02:19:01.393027 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:01.392972 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:05.392955922 +0000 UTC m=+68.878814889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : configmap "kube-rbac-proxy" not found Sep 05 02:19:02.700787 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:02.700747 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:19:02.701195 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:02.700803 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:19:02.703590 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:02.703571 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 05 02:19:02.703644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:02.703602 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 05 02:19:02.710997 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:02.710982 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 05 02:19:02.711079 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:02.711045 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs podName:257406f0-bf02-4fea-a10b-daf9750ddefc nodeName:}" failed. No retries permitted until 2026-09-05 02:20:06.711025616 +0000 UTC m=+130.196884572 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs") pod "network-metrics-daemon-r47jj" (UID: "257406f0-bf02-4fea-a10b-daf9750ddefc") : secret "metrics-daemon-secret" not found Sep 05 02:19:02.713202 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:02.713175 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d398890b-b9a3-4b54-abdd-d0f90f315b66-original-pull-secret\") pod \"global-pull-secret-syncer-74l8j\" (UID: \"d398890b-b9a3-4b54-abdd-d0f90f315b66\") " pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:19:02.727353 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:02.727336 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-74l8j" Sep 05 02:19:02.902740 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:02.902709 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:19:02.905364 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:02.905334 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th29q\" (UniqueName: \"kubernetes.io/projected/b5be9e27-b6e4-4477-9c67-2245faef62e9-kube-api-access-th29q\") pod \"network-check-target-jslqd\" (UID: \"b5be9e27-b6e4-4477-9c67-2245faef62e9\") " pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:19:02.998174 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:02.998145 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-74l8j"] Sep 05 02:19:03.001672 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:03.001639 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd398890b_b9a3_4b54_abdd_d0f90f315b66.slice/crio-8771b62cdd2d5c24a238e8f782ef505193ba1c97e58f52ff20820a99ca59b2a4 WatchSource:0}: Error finding container 8771b62cdd2d5c24a238e8f782ef505193ba1c97e58f52ff20820a99ca59b2a4: Status 404 returned error can't find the container with id 8771b62cdd2d5c24a238e8f782ef505193ba1c97e58f52ff20820a99ca59b2a4 Sep 05 02:19:03.024778 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.024760 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hk4wx\"" Sep 05 02:19:03.032658 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.032639 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:19:03.175100 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.175066 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:19:03.175259 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.175133 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:19:03.175477 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.175465 2567 scope.go:117] "RemoveContainer" containerID="5937dee3ed4552bfeaf46b5bfef1381d5f1ddfb6f2c1374992dcb977a743d774" Sep 05 02:19:03.175666 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:03.175648 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-cfg6p_openshift-console-operator(af6ef192-9048-4fb7-9b65-b2de6f8a7048)\"" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" podUID="af6ef192-9048-4fb7-9b65-b2de6f8a7048" Sep 05 02:19:03.298785 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.298710 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jslqd"] Sep 05 02:19:03.301835 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:03.301800 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5be9e27_b6e4_4477_9c67_2245faef62e9.slice/crio-83f8b3280125b0098291a04daaebc704b7d2e2fcd6a9685bd58b24059c5acb36 WatchSource:0}: Error finding container 83f8b3280125b0098291a04daaebc704b7d2e2fcd6a9685bd58b24059c5acb36: Status 404 returned error can't find the container with id 83f8b3280125b0098291a04daaebc704b7d2e2fcd6a9685bd58b24059c5acb36 Sep 05 02:19:03.426317 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.425921 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jslqd" event={"ID":"b5be9e27-b6e4-4477-9c67-2245faef62e9","Type":"ContainerStarted","Data":"a22ae2d6bd2127b5e8bd03c202d6e5cffa841b23a6c97800967e0e359757f111"} Sep 05 02:19:03.426317 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.425972 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jslqd" event={"ID":"b5be9e27-b6e4-4477-9c67-2245faef62e9","Type":"ContainerStarted","Data":"83f8b3280125b0098291a04daaebc704b7d2e2fcd6a9685bd58b24059c5acb36"} Sep 05 02:19:03.426317 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.426028 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:19:03.427410 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.427386 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-74l8j" event={"ID":"d398890b-b9a3-4b54-abdd-d0f90f315b66","Type":"ContainerStarted","Data":"8771b62cdd2d5c24a238e8f782ef505193ba1c97e58f52ff20820a99ca59b2a4"} Sep 05 02:19:03.446695 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:03.446640 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jslqd" podStartSLOduration=66.446620684 podStartE2EDuration="1m6.446620684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:03.44620327 +0000 UTC m=+66.932062246" watchObservedRunningTime="2026-09-05 02:19:03.446620684 +0000 UTC m=+66.932479660" Sep 05 02:19:04.213841 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:04.213807 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:19:04.214293 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:04.213963 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 02:19:04.214293 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:04.214034 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls podName:b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc nodeName:}" failed. No retries permitted until 2026-09-05 02:19:36.214016602 +0000 UTC m=+99.699875558 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls") pod "dns-default-ghw92" (UID: "b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc") : secret "dns-default-metrics-tls" not found Sep 05 02:19:05.425382 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:05.425345 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:19:05.425812 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:05.425414 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:19:05.425812 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:05.425530 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 02:19:05.425812 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:05.425532 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 02:19:05.425812 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:05.425591 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:13.425575961 +0000 UTC m=+76.911434918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : configmap "kube-rbac-proxy" not found Sep 05 02:19:05.425812 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:05.425604 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls podName:611d2010-264b-444d-8bf2-f856af766d89 nodeName:}" failed. No retries permitted until 2026-09-05 02:19:13.425598383 +0000 UTC m=+76.911457336 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hp58v" (UID: "611d2010-264b-444d-8bf2-f856af766d89") : secret "insights-runtime-extractor-tls" not found Sep 05 02:19:07.443797 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:07.443760 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-74l8j" event={"ID":"d398890b-b9a3-4b54-abdd-d0f90f315b66","Type":"ContainerStarted","Data":"d5bd11c12d725428adeb79db84ccf719eadf11f2ddf762f6335e7ed4735caa9a"} Sep 05 02:19:07.460722 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:07.460671 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-74l8j" podStartSLOduration=66.953665142 podStartE2EDuration="1m10.460655643s" totalImagesPullingTime="3.506990501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:07.459740492 +0000 UTC m=+70.945599471" watchObservedRunningTime="2026-09-05 02:19:07.460655643 +0000 UTC m=+70.946514618" Sep 05 02:19:08.653584 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.653525 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:19:08.654093 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.653685 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:19:08.656182 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.656155 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cb2ec8db-6ba7-4221-b886-3a679682581d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wn8qc\" (UID: \"cb2ec8db-6ba7-4221-b886-3a679682581d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:19:08.656288 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.656155 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9facd67b-62a7-4e19-b399-19b68d0d97a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lfv7f\" (UID: \"9facd67b-62a7-4e19-b399-19b68d0d97a4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:19:08.667192 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.667170 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5n7ln\"" Sep 05 02:19:08.673564 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.673545 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hfhzw\"" Sep 05 02:19:08.675309 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.675295 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" Sep 05 02:19:08.681537 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.681518 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" Sep 05 02:19:08.951927 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.951848 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc"] Sep 05 02:19:08.958746 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:08.958724 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f"] Sep 05 02:19:08.967438 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:08.967411 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9facd67b_62a7_4e19_b399_19b68d0d97a4.slice/crio-5cdf43fc26b036ffa4fac526dae759c0bb23a251a01292d0c078277546a98b61 WatchSource:0}: Error finding container 5cdf43fc26b036ffa4fac526dae759c0bb23a251a01292d0c078277546a98b61: Status 404 returned error can't find the container with id 5cdf43fc26b036ffa4fac526dae759c0bb23a251a01292d0c078277546a98b61 Sep 05 02:19:09.451308 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:09.451276 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" event={"ID":"cb2ec8db-6ba7-4221-b886-3a679682581d","Type":"ContainerStarted","Data":"a764e95e872623fb0b51c4c7488acef1030c98ca694719f98859fc331f1856a6"} Sep 05 02:19:09.452348 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:09.452325 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" event={"ID":"9facd67b-62a7-4e19-b399-19b68d0d97a4","Type":"ContainerStarted","Data":"5cdf43fc26b036ffa4fac526dae759c0bb23a251a01292d0c078277546a98b61"} Sep 05 02:19:11.461743 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:11.461700 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" event={"ID":"9facd67b-62a7-4e19-b399-19b68d0d97a4","Type":"ContainerStarted","Data":"de043a6fc57163508f33705e1bf518ee9b5e498f58a1e80eaae1169842ce5056"} Sep 05 02:19:11.484523 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:11.483730 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lfv7f" podStartSLOduration=17.734191384 podStartE2EDuration="19.483702779s" totalImagesPullingTime="1.749511395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:11.482328425 +0000 UTC m=+74.968187404" watchObservedRunningTime="2026-09-05 02:19:11.483702779 +0000 UTC m=+74.969561755" Sep 05 02:19:12.467386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:12.467351 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" event={"ID":"cb2ec8db-6ba7-4221-b886-3a679682581d","Type":"ContainerStarted","Data":"b8ed79f3f23955246a642cef5418825578cd13557b3edcd71bec9a334b709eac"} Sep 05 02:19:12.467898 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:12.467431 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" event={"ID":"cb2ec8db-6ba7-4221-b886-3a679682581d","Type":"ContainerStarted","Data":"e05fbaaaf6b5fae383ef459c3ec7edf3248aefbaa327866b505738bf036dfdb9"} Sep 05 02:19:12.501814 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:12.501767 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wn8qc" podStartSLOduration=17.515822635 podStartE2EDuration="20.501750811s" totalImagesPullingTime="2.985928176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:12.501188895 +0000 UTC m=+75.987047881" watchObservedRunningTime="2026-09-05 02:19:12.501750811 +0000 UTC m=+75.987609785" Sep 05 02:19:13.493441 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:13.493407 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:19:13.493912 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:13.493460 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:19:13.493960 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:13.493909 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/611d2010-264b-444d-8bf2-f856af766d89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:19:13.495856 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:13.495836 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/611d2010-264b-444d-8bf2-f856af766d89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hp58v\" (UID: \"611d2010-264b-444d-8bf2-f856af766d89\") " pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:19:13.556515 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:13.556463 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hp58v" Sep 05 02:19:13.838237 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:13.838143 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hp58v"] Sep 05 02:19:13.842222 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:13.842189 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611d2010_264b_444d_8bf2_f856af766d89.slice/crio-75268d054cbd56edefd696c4c8363ae72306f1150aad886be9b5fc56d274496a WatchSource:0}: Error finding container 75268d054cbd56edefd696c4c8363ae72306f1150aad886be9b5fc56d274496a: Status 404 returned error can't find the container with id 75268d054cbd56edefd696c4c8363ae72306f1150aad886be9b5fc56d274496a Sep 05 02:19:14.475597 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:14.475568 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hp58v" event={"ID":"611d2010-264b-444d-8bf2-f856af766d89","Type":"ContainerStarted","Data":"b542f9303d603249297393845bddeb3d4a8b26366352e2527410186bbd2ecaf0"} Sep 05 02:19:14.475774 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:14.475627 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hp58v" event={"ID":"611d2010-264b-444d-8bf2-f856af766d89","Type":"ContainerStarted","Data":"8d23dcf06942a14ae820c93e68f850aa229a10289a2ea43f7f5e994f4b824b34"} Sep 05 02:19:14.475774 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:14.475649 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hp58v" event={"ID":"611d2010-264b-444d-8bf2-f856af766d89","Type":"ContainerStarted","Data":"75268d054cbd56edefd696c4c8363ae72306f1150aad886be9b5fc56d274496a"} Sep 05 02:19:17.495766 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:17.495716 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hp58v" event={"ID":"611d2010-264b-444d-8bf2-f856af766d89","Type":"ContainerStarted","Data":"37283ad5177b6c661ef79aac8db68f104e9d3fdc3875542554b7b34978032a53"} Sep 05 02:19:17.513386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:17.513335 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hp58v" podStartSLOduration=17.424386208 podStartE2EDuration="20.513319676s" totalImagesPullingTime="3.088933468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 02:18:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:17.512724545 +0000 UTC m=+80.998583512" watchObservedRunningTime="2026-09-05 02:19:17.513319676 +0000 UTC m=+80.999178650" Sep 05 02:19:18.013096 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:18.013067 2567 scope.go:117] "RemoveContainer" containerID="5937dee3ed4552bfeaf46b5bfef1381d5f1ddfb6f2c1374992dcb977a743d774" Sep 05 02:19:18.500419 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:18.500391 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:19:18.500872 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:18.500854 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/1.log" Sep 05 02:19:18.500921 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:18.500890 2567 generic.go:356] "Generic (PLEG): container finished" podID="af6ef192-9048-4fb7-9b65-b2de6f8a7048" containerID="7de64a629aba71caa7ac59c83d062995fc10ec0ba91f6c28cb7cd287b369fde6" exitCode=255 Sep 05 02:19:18.500993 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:18.500973 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" event={"ID":"af6ef192-9048-4fb7-9b65-b2de6f8a7048","Type":"ContainerDied","Data":"7de64a629aba71caa7ac59c83d062995fc10ec0ba91f6c28cb7cd287b369fde6"} Sep 05 02:19:18.501057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:18.501043 2567 scope.go:117] "RemoveContainer" containerID="5937dee3ed4552bfeaf46b5bfef1381d5f1ddfb6f2c1374992dcb977a743d774" Sep 05 02:19:18.501366 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:18.501348 2567 scope.go:117] "RemoveContainer" containerID="7de64a629aba71caa7ac59c83d062995fc10ec0ba91f6c28cb7cd287b369fde6" Sep 05 02:19:18.501542 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:18.501528 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-cfg6p_openshift-console-operator(af6ef192-9048-4fb7-9b65-b2de6f8a7048)\"" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" podUID="af6ef192-9048-4fb7-9b65-b2de6f8a7048" Sep 05 02:19:19.506075 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:19.506046 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:19:22.682839 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.682804 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b"] Sep 05 02:19:22.687202 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.687178 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" Sep 05 02:19:22.689682 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.689661 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-cvxkl\"" Sep 05 02:19:22.689803 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.689661 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 05 02:19:22.690619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.690602 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 05 02:19:22.690672 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.690601 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 05 02:19:22.690709 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.690676 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 05 02:19:22.703697 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.703673 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b"] Sep 05 02:19:22.757033 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.756994 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggcpd\" (UniqueName: \"kubernetes.io/projected/e06eb131-3c21-450a-b2b5-133c498c5824-kube-api-access-ggcpd\") pod \"managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b\" (UID: \"e06eb131-3c21-450a-b2b5-133c498c5824\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" Sep 05 02:19:22.757228 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.757048 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e06eb131-3c21-450a-b2b5-133c498c5824-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b\" (UID: \"e06eb131-3c21-450a-b2b5-133c498c5824\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" Sep 05 02:19:22.769820 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.769778 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx"] Sep 05 02:19:22.772924 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.772906 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" Sep 05 02:19:22.775820 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.775802 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 05 02:19:22.775897 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.775853 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-l4ns7\"" Sep 05 02:19:22.786962 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.786939 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx"] Sep 05 02:19:22.830464 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.830430 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66cfff688c-45r6r"] Sep 05 02:19:22.833450 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.833435 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.835992 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.835968 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 05 02:19:22.836122 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.835989 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 05 02:19:22.836174 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.836128 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f79x9\"" Sep 05 02:19:22.839080 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.839059 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 05 02:19:22.843311 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.843292 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 05 02:19:22.853613 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.853591 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66cfff688c-45r6r"] Sep 05 02:19:22.858427 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858408 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/120f4523-946f-45dd-ba90-8ade102169f3-trusted-ca\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.858548 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858434 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/120f4523-946f-45dd-ba90-8ade102169f3-installation-pull-secrets\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.858548 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858452 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/120f4523-946f-45dd-ba90-8ade102169f3-bound-sa-token\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.858548 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c9f798c1-bd10-4bce-b33d-b12ac31ee286-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-dbbjx\" (UID: \"c9f798c1-bd10-4bce-b33d-b12ac31ee286\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" Sep 05 02:19:22.858709 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858625 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcpd\" (UniqueName: \"kubernetes.io/projected/e06eb131-3c21-450a-b2b5-133c498c5824-kube-api-access-ggcpd\") pod \"managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b\" (UID: \"e06eb131-3c21-450a-b2b5-133c498c5824\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" Sep 05 02:19:22.858709 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858672 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/120f4523-946f-45dd-ba90-8ade102169f3-registry-certificates\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.858809 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858709 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/120f4523-946f-45dd-ba90-8ade102169f3-image-registry-private-configuration\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.858809 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858741 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e06eb131-3c21-450a-b2b5-133c498c5824-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b\" (UID: \"e06eb131-3c21-450a-b2b5-133c498c5824\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" Sep 05 02:19:22.858809 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858795 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/120f4523-946f-45dd-ba90-8ade102169f3-registry-tls\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.858945 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858834 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/120f4523-946f-45dd-ba90-8ade102169f3-ca-trust-extracted\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.858945 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.858860 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr54s\" (UniqueName: \"kubernetes.io/projected/120f4523-946f-45dd-ba90-8ade102169f3-kube-api-access-rr54s\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.861302 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.861278 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e06eb131-3c21-450a-b2b5-133c498c5824-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b\" (UID: \"e06eb131-3c21-450a-b2b5-133c498c5824\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" Sep 05 02:19:22.874016 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.873993 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggcpd\" (UniqueName: \"kubernetes.io/projected/e06eb131-3c21-450a-b2b5-133c498c5824-kube-api-access-ggcpd\") pod \"managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b\" (UID: \"e06eb131-3c21-450a-b2b5-133c498c5824\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" Sep 05 02:19:22.959957 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.959931 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/120f4523-946f-45dd-ba90-8ade102169f3-registry-certificates\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.960110 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.959964 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/120f4523-946f-45dd-ba90-8ade102169f3-image-registry-private-configuration\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.960151 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.960111 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/120f4523-946f-45dd-ba90-8ade102169f3-registry-tls\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.960189 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.960152 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/120f4523-946f-45dd-ba90-8ade102169f3-ca-trust-extracted\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.960189 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.960178 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr54s\" (UniqueName: \"kubernetes.io/projected/120f4523-946f-45dd-ba90-8ade102169f3-kube-api-access-rr54s\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.960247 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.960223 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/120f4523-946f-45dd-ba90-8ade102169f3-trusted-ca\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.960293 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.960250 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/120f4523-946f-45dd-ba90-8ade102169f3-installation-pull-secrets\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.960293 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.960276 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/120f4523-946f-45dd-ba90-8ade102169f3-bound-sa-token\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.960392 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.960313 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c9f798c1-bd10-4bce-b33d-b12ac31ee286-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-dbbjx\" (UID: \"c9f798c1-bd10-4bce-b33d-b12ac31ee286\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" Sep 05 02:19:22.960894 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.960861 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/120f4523-946f-45dd-ba90-8ade102169f3-registry-certificates\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.961186 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.961136 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/120f4523-946f-45dd-ba90-8ade102169f3-ca-trust-extracted\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.962106 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.962078 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/120f4523-946f-45dd-ba90-8ade102169f3-trusted-ca\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.962802 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.962781 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/120f4523-946f-45dd-ba90-8ade102169f3-registry-tls\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.962866 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.962851 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/120f4523-946f-45dd-ba90-8ade102169f3-image-registry-private-configuration\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.962954 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.962931 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c9f798c1-bd10-4bce-b33d-b12ac31ee286-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-dbbjx\" (UID: \"c9f798c1-bd10-4bce-b33d-b12ac31ee286\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" Sep 05 02:19:22.963470 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.963453 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/120f4523-946f-45dd-ba90-8ade102169f3-installation-pull-secrets\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.971671 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.971650 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr54s\" (UniqueName: \"kubernetes.io/projected/120f4523-946f-45dd-ba90-8ade102169f3-kube-api-access-rr54s\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:22.972277 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:22.972257 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/120f4523-946f-45dd-ba90-8ade102169f3-bound-sa-token\") pod \"image-registry-66cfff688c-45r6r\" (UID: \"120f4523-946f-45dd-ba90-8ade102169f3\") " pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:23.008389 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.008363 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" Sep 05 02:19:23.081944 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.081906 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" Sep 05 02:19:23.142848 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.142821 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:23.175049 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.175015 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:19:23.175220 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.175078 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:19:23.175457 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.175441 2567 scope.go:117] "RemoveContainer" containerID="7de64a629aba71caa7ac59c83d062995fc10ec0ba91f6c28cb7cd287b369fde6" Sep 05 02:19:23.175666 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:23.175645 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-cfg6p_openshift-console-operator(af6ef192-9048-4fb7-9b65-b2de6f8a7048)\"" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" podUID="af6ef192-9048-4fb7-9b65-b2de6f8a7048" Sep 05 02:19:23.279807 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.279733 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b"] Sep 05 02:19:23.282802 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:23.282774 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode06eb131_3c21_450a_b2b5_133c498c5824.slice/crio-24c53fc2eb87642f81a1474bccc6d96341221d7ab69bf433faf6fccd2da357e4 WatchSource:0}: Error finding container 24c53fc2eb87642f81a1474bccc6d96341221d7ab69bf433faf6fccd2da357e4: Status 404 returned error can't find the container with id 24c53fc2eb87642f81a1474bccc6d96341221d7ab69bf433faf6fccd2da357e4 Sep 05 02:19:23.354424 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.354400 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx"] Sep 05 02:19:23.356764 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:23.356740 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f798c1_bd10_4bce_b33d_b12ac31ee286.slice/crio-474060caf44cfdd0239b9b8f6b39437d5ed9c9c5a90b03bd209d469984a260eb WatchSource:0}: Error finding container 474060caf44cfdd0239b9b8f6b39437d5ed9c9c5a90b03bd209d469984a260eb: Status 404 returned error can't find the container with id 474060caf44cfdd0239b9b8f6b39437d5ed9c9c5a90b03bd209d469984a260eb Sep 05 02:19:23.415325 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.415296 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66cfff688c-45r6r"] Sep 05 02:19:23.418309 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:23.418275 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120f4523_946f_45dd_ba90_8ade102169f3.slice/crio-8fa38b466421ff8a668090231cc0281a1a3c6243f9cd5e48cc8bb3be9d87e331 WatchSource:0}: Error finding container 8fa38b466421ff8a668090231cc0281a1a3c6243f9cd5e48cc8bb3be9d87e331: Status 404 returned error can't find the container with id 8fa38b466421ff8a668090231cc0281a1a3c6243f9cd5e48cc8bb3be9d87e331 Sep 05 02:19:23.520479 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.520446 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" event={"ID":"e06eb131-3c21-450a-b2b5-133c498c5824","Type":"ContainerStarted","Data":"24c53fc2eb87642f81a1474bccc6d96341221d7ab69bf433faf6fccd2da357e4"} Sep 05 02:19:23.522013 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.521983 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66cfff688c-45r6r" event={"ID":"120f4523-946f-45dd-ba90-8ade102169f3","Type":"ContainerStarted","Data":"3c43c837630028573bb522e3459c374e9bd175257a0eab758549cd5976229f5d"} Sep 05 02:19:23.522146 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.522041 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66cfff688c-45r6r" event={"ID":"120f4523-946f-45dd-ba90-8ade102169f3","Type":"ContainerStarted","Data":"8fa38b466421ff8a668090231cc0281a1a3c6243f9cd5e48cc8bb3be9d87e331"} Sep 05 02:19:23.522146 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.522074 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:23.523281 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.523259 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" event={"ID":"c9f798c1-bd10-4bce-b33d-b12ac31ee286","Type":"ContainerStarted","Data":"474060caf44cfdd0239b9b8f6b39437d5ed9c9c5a90b03bd209d469984a260eb"} Sep 05 02:19:23.540995 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:23.540904 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66cfff688c-45r6r" podStartSLOduration=1.540884738 podStartE2EDuration="1.540884738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:23.539785683 +0000 UTC m=+87.025644658" watchObservedRunningTime="2026-09-05 02:19:23.540884738 +0000 UTC m=+87.026743716" Sep 05 02:19:24.528648 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:24.528612 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" event={"ID":"c9f798c1-bd10-4bce-b33d-b12ac31ee286","Type":"ContainerStarted","Data":"cbd1038fc1663f8681a264c48f788cf574b5b20bbb478f671e96b31ff5cc4310"} Sep 05 02:19:24.543424 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:24.543367 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" podStartSLOduration=1.468127034 podStartE2EDuration="2.543348733s" totalImagesPullingTime="1.075221699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:24.543257114 +0000 UTC m=+88.029116092" watchObservedRunningTime="2026-09-05 02:19:24.543348733 +0000 UTC m=+88.029207709" Sep 05 02:19:25.533151 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:25.533115 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" Sep 05 02:19:25.538808 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:25.538779 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-dbbjx" Sep 05 02:19:26.433356 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.433318 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-hgh8g"] Sep 05 02:19:26.436633 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.436613 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.439800 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.439765 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 05 02:19:26.439907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.439803 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 05 02:19:26.439907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.439829 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lfkbk\"" Sep 05 02:19:26.439907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.439808 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 05 02:19:26.446103 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.446083 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-hgh8g"] Sep 05 02:19:26.489296 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.489266 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09810df9-1b98-489b-89a8-a00f04328c4a-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.489480 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.489327 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09810df9-1b98-489b-89a8-a00f04328c4a-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.489480 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.489350 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09810df9-1b98-489b-89a8-a00f04328c4a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.489480 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.489442 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss4z6\" (UniqueName: \"kubernetes.io/projected/09810df9-1b98-489b-89a8-a00f04328c4a-kube-api-access-ss4z6\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.590135 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.590101 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss4z6\" (UniqueName: \"kubernetes.io/projected/09810df9-1b98-489b-89a8-a00f04328c4a-kube-api-access-ss4z6\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.590537 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.590166 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09810df9-1b98-489b-89a8-a00f04328c4a-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.590537 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.590213 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09810df9-1b98-489b-89a8-a00f04328c4a-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.590537 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.590230 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09810df9-1b98-489b-89a8-a00f04328c4a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.590537 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:26.590349 2567 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 05 02:19:26.590537 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:26.590417 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09810df9-1b98-489b-89a8-a00f04328c4a-prometheus-operator-tls podName:09810df9-1b98-489b-89a8-a00f04328c4a nodeName:}" failed. No retries permitted until 2026-09-05 02:19:27.090396487 +0000 UTC m=+90.576255441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/09810df9-1b98-489b-89a8-a00f04328c4a-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-hgh8g" (UID: "09810df9-1b98-489b-89a8-a00f04328c4a") : secret "prometheus-operator-tls" not found Sep 05 02:19:26.590819 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.590800 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09810df9-1b98-489b-89a8-a00f04328c4a-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.592662 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.592638 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09810df9-1b98-489b-89a8-a00f04328c4a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:26.601358 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:26.601338 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss4z6\" (UniqueName: \"kubernetes.io/projected/09810df9-1b98-489b-89a8-a00f04328c4a-kube-api-access-ss4z6\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:27.094171 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:27.094125 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09810df9-1b98-489b-89a8-a00f04328c4a-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:27.096654 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:27.096636 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09810df9-1b98-489b-89a8-a00f04328c4a-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-hgh8g\" (UID: \"09810df9-1b98-489b-89a8-a00f04328c4a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:27.346661 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:27.346545 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" Sep 05 02:19:27.541803 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:27.541768 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" event={"ID":"e06eb131-3c21-450a-b2b5-133c498c5824","Type":"ContainerStarted","Data":"97a24af4052b34f58396eca250b079aae26764de3414f4c43174e828abc304d6"} Sep 05 02:19:27.564091 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:27.564044 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7476b9b6d4-5pb9b" podStartSLOduration=2.204346815 podStartE2EDuration="5.564023344s" totalImagesPullingTime="3.359676529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:27.563818065 +0000 UTC m=+91.049677041" watchObservedRunningTime="2026-09-05 02:19:27.564023344 +0000 UTC m=+91.049882320" Sep 05 02:19:27.614475 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:27.614413 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-hgh8g"] Sep 05 02:19:27.617560 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:27.617539 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09810df9_1b98_489b_89a8_a00f04328c4a.slice/crio-2f481584abfafb0fd457ae8433cfa01c06c8fb90fa95aeb4308d025ca3eb27d7 WatchSource:0}: Error finding container 2f481584abfafb0fd457ae8433cfa01c06c8fb90fa95aeb4308d025ca3eb27d7: Status 404 returned error can't find the container with id 2f481584abfafb0fd457ae8433cfa01c06c8fb90fa95aeb4308d025ca3eb27d7 Sep 05 02:19:28.551830 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:28.551797 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" event={"ID":"09810df9-1b98-489b-89a8-a00f04328c4a","Type":"ContainerStarted","Data":"2f481584abfafb0fd457ae8433cfa01c06c8fb90fa95aeb4308d025ca3eb27d7"} Sep 05 02:19:29.556847 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:29.556812 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" event={"ID":"09810df9-1b98-489b-89a8-a00f04328c4a","Type":"ContainerStarted","Data":"2d57796740ed5cb28b94cfa7f68fe695fa2c08db34e4d30c786e499647770ef7"} Sep 05 02:19:29.557225 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:29.556862 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" event={"ID":"09810df9-1b98-489b-89a8-a00f04328c4a","Type":"ContainerStarted","Data":"4ce316e5002e197293ea9af32d84cc4e4e86f56d4f59e8494bfb67070044f680"} Sep 05 02:19:29.574600 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:29.574541 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-hgh8g" podStartSLOduration=2.215182592 podStartE2EDuration="3.574522178s" totalImagesPullingTime="1.359339586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:29.574176218 +0000 UTC m=+93.060035194" watchObservedRunningTime="2026-09-05 02:19:29.574522178 +0000 UTC m=+93.060381150" Sep 05 02:19:32.256438 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.256401 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4wqbc"] Sep 05 02:19:32.281525 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.281479 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.284580 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.284553 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 05 02:19:32.284706 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.284554 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pzst5\"" Sep 05 02:19:32.284706 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.284615 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 05 02:19:32.285373 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.285356 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 05 02:19:32.340749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.340712 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.340749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.340747 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0361429b-11bd-4c00-814e-ee5bb97d77a8-root\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.340938 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.340771 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-wtmp\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.340938 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.340882 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-textfile\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.340938 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.340906 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0361429b-11bd-4c00-814e-ee5bb97d77a8-sys\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.340938 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.340923 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbcp2\" (UniqueName: \"kubernetes.io/projected/0361429b-11bd-4c00-814e-ee5bb97d77a8-kube-api-access-mbcp2\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.341145 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.340982 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0361429b-11bd-4c00-814e-ee5bb97d77a8-metrics-client-ca\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.341145 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.341003 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.341145 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.341053 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-tls\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.441809 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.441774 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-textfile\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.441809 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.441810 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0361429b-11bd-4c00-814e-ee5bb97d77a8-sys\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.441829 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbcp2\" (UniqueName: \"kubernetes.io/projected/0361429b-11bd-4c00-814e-ee5bb97d77a8-kube-api-access-mbcp2\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.441851 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0361429b-11bd-4c00-814e-ee5bb97d77a8-metrics-client-ca\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.441876 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.441900 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0361429b-11bd-4c00-814e-ee5bb97d77a8-sys\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.441915 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-tls\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.441970 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.442002 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0361429b-11bd-4c00-814e-ee5bb97d77a8-root\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.442035 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-wtmp\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442410 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.442107 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0361429b-11bd-4c00-814e-ee5bb97d77a8-root\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442410 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.442195 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-wtmp\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442410 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.442380 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-textfile\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442598 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.442584 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.442689 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.442670 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0361429b-11bd-4c00-814e-ee5bb97d77a8-metrics-client-ca\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.444482 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.444461 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-tls\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.444582 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.444524 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0361429b-11bd-4c00-814e-ee5bb97d77a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.456684 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.456667 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbcp2\" (UniqueName: \"kubernetes.io/projected/0361429b-11bd-4c00-814e-ee5bb97d77a8-kube-api-access-mbcp2\") pod \"node-exporter-4wqbc\" (UID: \"0361429b-11bd-4c00-814e-ee5bb97d77a8\") " pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.591189 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:32.591117 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4wqbc" Sep 05 02:19:32.611137 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:32.611099 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0361429b_11bd_4c00_814e_ee5bb97d77a8.slice/crio-3c87852854d52838ac01f1b929873cf04d76a81ab539ed67543977ae049bb714 WatchSource:0}: Error finding container 3c87852854d52838ac01f1b929873cf04d76a81ab539ed67543977ae049bb714: Status 404 returned error can't find the container with id 3c87852854d52838ac01f1b929873cf04d76a81ab539ed67543977ae049bb714 Sep 05 02:19:33.573147 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:33.573118 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4wqbc" event={"ID":"0361429b-11bd-4c00-814e-ee5bb97d77a8","Type":"ContainerStarted","Data":"3c87852854d52838ac01f1b929873cf04d76a81ab539ed67543977ae049bb714"} Sep 05 02:19:34.434849 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:34.434820 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jslqd" Sep 05 02:19:34.577908 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:34.577873 2567 generic.go:356] "Generic (PLEG): container finished" podID="0361429b-11bd-4c00-814e-ee5bb97d77a8" containerID="c8b7f623f6dfcc817e97171faf57f099b5cc60fe8ceede06c0a9a0de94f4c259" exitCode=0 Sep 05 02:19:34.578385 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:34.577914 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4wqbc" event={"ID":"0361429b-11bd-4c00-814e-ee5bb97d77a8","Type":"ContainerDied","Data":"c8b7f623f6dfcc817e97171faf57f099b5cc60fe8ceede06c0a9a0de94f4c259"} Sep 05 02:19:35.584426 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:35.584394 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4wqbc" event={"ID":"0361429b-11bd-4c00-814e-ee5bb97d77a8","Type":"ContainerStarted","Data":"2547075cb7077ae322ec66b3ac9f4c54b297cf7114b1b2a779791311b4054385"} Sep 05 02:19:35.584975 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:35.584945 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4wqbc" event={"ID":"0361429b-11bd-4c00-814e-ee5bb97d77a8","Type":"ContainerStarted","Data":"e11fa30e39874f6cfb1e6debc9d2d5b59f1a82240e0f7759c4fc3911779fb287"} Sep 05 02:19:35.606782 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:35.606739 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4wqbc" podStartSLOduration=2.618143718 podStartE2EDuration="3.606724886s" totalImagesPullingTime="918.431164ms" totalInitContainerRuntime="70.150004ms" isStatefulPod=true podCreationTimestamp="2026-09-05 02:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:35.60554797 +0000 UTC m=+99.091406938" watchObservedRunningTime="2026-09-05 02:19:35.606724886 +0000 UTC m=+99.092583882" Sep 05 02:19:36.275012 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:36.274971 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:19:36.277398 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:36.277380 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc-metrics-tls\") pod \"dns-default-ghw92\" (UID: \"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc\") " pod="openshift-dns/dns-default-ghw92" Sep 05 02:19:36.573132 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:36.573060 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cl7gt\"" Sep 05 02:19:36.581271 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:36.581248 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ghw92" Sep 05 02:19:36.852544 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:36.852446 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ghw92"] Sep 05 02:19:36.856350 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:36.856324 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f5d9a6_4ee3_41c4_9be0_c57ed41451cc.slice/crio-90ed28a7be1b66c0ba0945a5a90d92e75ceb064571eb35e51b8afd65707037fd WatchSource:0}: Error finding container 90ed28a7be1b66c0ba0945a5a90d92e75ceb064571eb35e51b8afd65707037fd: Status 404 returned error can't find the container with id 90ed28a7be1b66c0ba0945a5a90d92e75ceb064571eb35e51b8afd65707037fd Sep 05 02:19:37.014852 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:37.014806 2567 scope.go:117] "RemoveContainer" containerID="7de64a629aba71caa7ac59c83d062995fc10ec0ba91f6c28cb7cd287b369fde6" Sep 05 02:19:37.015062 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:19:37.015024 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-cfg6p_openshift-console-operator(af6ef192-9048-4fb7-9b65-b2de6f8a7048)\"" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" podUID="af6ef192-9048-4fb7-9b65-b2de6f8a7048" Sep 05 02:19:37.593234 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:37.593192 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ghw92" event={"ID":"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc","Type":"ContainerStarted","Data":"90ed28a7be1b66c0ba0945a5a90d92e75ceb064571eb35e51b8afd65707037fd"} Sep 05 02:19:38.242894 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.242857 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 05 02:19:38.264210 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.264171 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.271353 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.271325 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 05 02:19:38.273753 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.273562 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 05 02:19:38.273895 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.273872 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 05 02:19:38.274115 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.274097 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 05 02:19:38.276718 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.276558 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 05 02:19:38.276849 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.276763 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 05 02:19:38.276959 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.276941 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 05 02:19:38.277349 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.277250 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 05 02:19:38.282701 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.277934 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 05 02:19:38.282701 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.278227 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 05 02:19:38.282701 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.278421 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 05 02:19:38.282701 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.279747 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 05 02:19:38.282701 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.281545 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 05 02:19:38.282701 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.281738 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 05 02:19:38.282701 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.281962 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-vvnr7c88qkli\"" Sep 05 02:19:38.283714 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.283512 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 05 02:19:38.283714 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.283567 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7zppx\"" Sep 05 02:19:38.287165 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.287146 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 05 02:19:38.396078 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396046 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396078 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396080 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396292 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396102 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396292 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396123 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396292 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396177 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396292 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396235 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396292 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396269 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396292 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396291 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396477 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396306 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396477 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396326 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396477 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396400 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396477 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396449 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-web-config\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396477 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396470 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396664 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396510 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396664 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396532 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396664 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396549 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-config-out\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396664 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396571 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-config\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396664 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396602 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396664 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396618 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.396664 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.396635 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb9cr\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-kube-api-access-lb9cr\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497095 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497022 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497095 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497061 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497095 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497083 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497351 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497108 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497351 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497132 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497351 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497279 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497351 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497323 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497561 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497358 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497561 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497388 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497561 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497427 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497561 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497466 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497769 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497561 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-web-config\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497769 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497769 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497616 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497769 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497643 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497769 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497690 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-config-out\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497769 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497720 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-config\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497769 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497729 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497769 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497743 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.497769 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497769 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.498170 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497795 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb9cr\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-kube-api-access-lb9cr\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.498170 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.497880 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.498170 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.498071 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.498356 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.498330 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.502231 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.498801 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.502231 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.501135 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.502231 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.501855 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.502231 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.501878 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.502231 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.501890 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.502592 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.502397 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.502592 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.502515 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.502957 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.502929 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.503847 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.503804 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.505303 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.505226 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.505303 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.505253 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-config-out\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.505478 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.505413 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.505669 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.505648 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-web-config\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.505836 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.505815 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-config\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.506951 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.506920 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.514025 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.513994 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb9cr\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-kube-api-access-lb9cr\") pod \"prometheus-k8s-0\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.576789 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.576755 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:38.885712 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:38.885680 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 05 02:19:38.889246 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:19:38.889220 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf09cdf_6abc_420e_b461_8474f191aa61.slice/crio-d7323dc8e8a8f2ea58559d935a8f177bc59ff3cb5130845d920e8bf60d0c8ff4 WatchSource:0}: Error finding container d7323dc8e8a8f2ea58559d935a8f177bc59ff3cb5130845d920e8bf60d0c8ff4: Status 404 returned error can't find the container with id d7323dc8e8a8f2ea58559d935a8f177bc59ff3cb5130845d920e8bf60d0c8ff4 Sep 05 02:19:39.602722 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:39.602683 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ghw92" event={"ID":"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc","Type":"ContainerStarted","Data":"547ae4e3bf3671256e3aaa012dbe2ddc5418f81c7d5b5bd83bdeb54e1e1c8abc"} Sep 05 02:19:39.603184 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:39.602762 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ghw92" Sep 05 02:19:39.603184 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:39.602789 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ghw92" event={"ID":"b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc","Type":"ContainerStarted","Data":"519cb1907bd0c5fc8ef5a95648f6d111dbe8f6a8bda180ed1273971f72310399"} Sep 05 02:19:39.604045 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:39.604023 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerStarted","Data":"d7323dc8e8a8f2ea58559d935a8f177bc59ff3cb5130845d920e8bf60d0c8ff4"} Sep 05 02:19:39.624574 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:39.624524 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ghw92" podStartSLOduration=65.653399234 podStartE2EDuration="1m7.624510014s" totalImagesPullingTime="1.97111078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:39.623530272 +0000 UTC m=+103.109389246" watchObservedRunningTime="2026-09-05 02:19:39.624510014 +0000 UTC m=+103.110368981" Sep 05 02:19:41.613617 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:41.613586 2567 generic.go:356] "Generic (PLEG): container finished" podID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerID="8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7" exitCode=0 Sep 05 02:19:41.613978 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:41.613627 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerDied","Data":"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7"} Sep 05 02:19:44.533960 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:44.533925 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66cfff688c-45r6r" Sep 05 02:19:45.634290 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:45.634251 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerStarted","Data":"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d"} Sep 05 02:19:45.634684 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:45.634324 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerStarted","Data":"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c"} Sep 05 02:19:46.641931 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:46.641858 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerStarted","Data":"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535"} Sep 05 02:19:46.642279 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:46.641931 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerStarted","Data":"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41"} Sep 05 02:19:46.642279 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:46.641995 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerStarted","Data":"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4"} Sep 05 02:19:46.642279 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:46.642034 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerStarted","Data":"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569"} Sep 05 02:19:46.673042 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:46.672992 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.230978444 podStartE2EDuration="8.672971997s" totalImagesPullingTime="6.422723062s" totalInitContainerRuntime="19.270491ms" isStatefulPod=false podCreationTimestamp="2026-09-05 02:19:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:46.671334135 +0000 UTC m=+110.157193108" watchObservedRunningTime="2026-09-05 02:19:46.672971997 +0000 UTC m=+110.158830972" Sep 05 02:19:48.577428 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:48.577392 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:19:49.610782 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:49.610751 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ghw92" Sep 05 02:19:52.012928 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:52.012899 2567 scope.go:117] "RemoveContainer" containerID="7de64a629aba71caa7ac59c83d062995fc10ec0ba91f6c28cb7cd287b369fde6" Sep 05 02:19:52.665692 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:52.665666 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:19:52.665853 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:52.665746 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" event={"ID":"af6ef192-9048-4fb7-9b65-b2de6f8a7048","Type":"ContainerStarted","Data":"667a1ed7c7f6d1a25586a248ba4ed68d4105b0008db34b5cd46cfa6649928ca0"} Sep 05 02:19:52.666034 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:52.666014 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:19:52.689973 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:52.689919 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" podStartSLOduration=57.593943534 podStartE2EDuration="1m0.689906034s" totalImagesPullingTime="3.0959625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:19:52.688811862 +0000 UTC m=+116.174670847" watchObservedRunningTime="2026-09-05 02:19:52.689906034 +0000 UTC m=+116.175765008" Sep 05 02:19:52.763836 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:19:52.763813 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-cfg6p" Sep 05 02:20:06.731981 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:06.731946 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:20:06.734359 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:06.734338 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/257406f0-bf02-4fea-a10b-daf9750ddefc-metrics-certs\") pod \"network-metrics-daemon-r47jj\" (UID: \"257406f0-bf02-4fea-a10b-daf9750ddefc\") " pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:20:07.025808 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.025728 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4nzpg\"" Sep 05 02:20:07.034051 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.034029 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r47jj" Sep 05 02:20:07.306988 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.306913 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r47jj"] Sep 05 02:20:07.310559 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:20:07.310532 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod257406f0_bf02_4fea_a10b_daf9750ddefc.slice/crio-b9b01b6dfce2babc206357a3e0054bee4e7fb8d7cfbcd0ba640309867a38b993 WatchSource:0}: Error finding container b9b01b6dfce2babc206357a3e0054bee4e7fb8d7cfbcd0ba640309867a38b993: Status 404 returned error can't find the container with id b9b01b6dfce2babc206357a3e0054bee4e7fb8d7cfbcd0ba640309867a38b993 Sep 05 02:20:07.721246 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.721212 2567 generic.go:356] "Generic (PLEG): container finished" podID="0b4e3d1f-b577-414a-a0e4-882b83ae9d45" containerID="421bbe3b4c99100ae01752c4af12976829711e92941358854c0636a665f93c69" exitCode=0 Sep 05 02:20:07.721424 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.721292 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" event={"ID":"0b4e3d1f-b577-414a-a0e4-882b83ae9d45","Type":"ContainerDied","Data":"421bbe3b4c99100ae01752c4af12976829711e92941358854c0636a665f93c69"} Sep 05 02:20:07.721714 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.721693 2567 scope.go:117] "RemoveContainer" containerID="421bbe3b4c99100ae01752c4af12976829711e92941358854c0636a665f93c69" Sep 05 02:20:07.723343 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.723320 2567 generic.go:356] "Generic (PLEG): container finished" podID="0b525226-203f-4f6f-91ed-93f07eb5653f" containerID="313762c7d3ff7431ce7bf97f353a8f49c1e623954190bd1a169d0bd34eaa7bf4" exitCode=0 Sep 05 02:20:07.723443 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.723349 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" event={"ID":"0b525226-203f-4f6f-91ed-93f07eb5653f","Type":"ContainerDied","Data":"313762c7d3ff7431ce7bf97f353a8f49c1e623954190bd1a169d0bd34eaa7bf4"} Sep 05 02:20:07.723720 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.723702 2567 scope.go:117] "RemoveContainer" containerID="313762c7d3ff7431ce7bf97f353a8f49c1e623954190bd1a169d0bd34eaa7bf4" Sep 05 02:20:07.724873 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:07.724854 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r47jj" event={"ID":"257406f0-bf02-4fea-a10b-daf9750ddefc","Type":"ContainerStarted","Data":"b9b01b6dfce2babc206357a3e0054bee4e7fb8d7cfbcd0ba640309867a38b993"} Sep 05 02:20:08.730779 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:08.730744 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r47jj" event={"ID":"257406f0-bf02-4fea-a10b-daf9750ddefc","Type":"ContainerStarted","Data":"48cd02a177c57cff90846fc5aabe4e5403e129bde5d0ac9e1b8dee99f5fedbf0"} Sep 05 02:20:08.731244 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:08.730807 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r47jj" event={"ID":"257406f0-bf02-4fea-a10b-daf9750ddefc","Type":"ContainerStarted","Data":"fd739cc43e4600ff881706b17532bb333832b95472b580a0e2e1c72c6d6baa15"} Sep 05 02:20:08.732655 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:08.732631 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-rr89z" event={"ID":"0b4e3d1f-b577-414a-a0e4-882b83ae9d45","Type":"ContainerStarted","Data":"f83af73aeea3248ced089eba86a7f20fd4836f5ffe2cc0136bdbe632dbafb985"} Sep 05 02:20:08.734438 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:08.734413 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-92bgx" event={"ID":"0b525226-203f-4f6f-91ed-93f07eb5653f","Type":"ContainerStarted","Data":"b69931b745d9de591545d12d79fd98bd3737c1b64ed68b5c9fe8549b1e64bc27"} Sep 05 02:20:08.770664 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:08.770616 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r47jj" podStartSLOduration=130.886611994 podStartE2EDuration="2m11.77060006s" totalImagesPullingTime="883.988066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:20:08.7699502 +0000 UTC m=+132.255809188" watchObservedRunningTime="2026-09-05 02:20:08.77060006 +0000 UTC m=+132.256459034" Sep 05 02:20:38.577207 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:38.577171 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:38.597192 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:38.597162 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:38.863587 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:38.863512 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:56.582822 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.582728 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 05 02:20:56.583302 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.583130 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="prometheus" containerID="cri-o://ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" gracePeriod=600 Sep 05 02:20:56.583302 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.583188 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy-web" containerID="cri-o://06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" gracePeriod=600 Sep 05 02:20:56.583302 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.583235 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="config-reloader" containerID="cri-o://3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" gracePeriod=600 Sep 05 02:20:56.583523 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.583188 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy-thanos" containerID="cri-o://0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" gracePeriod=600 Sep 05 02:20:56.583523 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.583236 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy" containerID="cri-o://0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" gracePeriod=600 Sep 05 02:20:56.583523 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.583175 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="thanos-sidecar" containerID="cri-o://282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" gracePeriod=600 Sep 05 02:20:56.833052 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.832992 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:56.919735 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.919698 2567 generic.go:356] "Generic (PLEG): container finished" podID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerID="0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" exitCode=0 Sep 05 02:20:56.919735 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.919730 2567 generic.go:356] "Generic (PLEG): container finished" podID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerID="0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" exitCode=0 Sep 05 02:20:56.919735 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.919737 2567 generic.go:356] "Generic (PLEG): container finished" podID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerID="06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" exitCode=0 Sep 05 02:20:56.919735 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.919744 2567 generic.go:356] "Generic (PLEG): container finished" podID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerID="282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" exitCode=0 Sep 05 02:20:56.920071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.919751 2567 generic.go:356] "Generic (PLEG): container finished" podID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerID="3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" exitCode=0 Sep 05 02:20:56.920071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.919758 2567 generic.go:356] "Generic (PLEG): container finished" podID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerID="ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" exitCode=0 Sep 05 02:20:56.920071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.919801 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:56.920071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.919792 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerDied","Data":"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535"} Sep 05 02:20:56.920071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.919971 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerDied","Data":"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41"} Sep 05 02:20:56.920071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.920019 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerDied","Data":"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4"} Sep 05 02:20:56.920071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.920059 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerDied","Data":"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569"} Sep 05 02:20:56.920365 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.920044 2567 scope.go:117] "RemoveContainer" containerID="0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" Sep 05 02:20:56.920365 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.920100 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerDied","Data":"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d"} Sep 05 02:20:56.920365 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.920142 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerDied","Data":"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c"} Sep 05 02:20:56.920365 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.920216 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3cf09cdf-6abc-420e-b461-8474f191aa61","Type":"ContainerDied","Data":"d7323dc8e8a8f2ea58559d935a8f177bc59ff3cb5130845d920e8bf60d0c8ff4"} Sep 05 02:20:56.928777 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.928756 2567 scope.go:117] "RemoveContainer" containerID="0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" Sep 05 02:20:56.935900 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.935883 2567 scope.go:117] "RemoveContainer" containerID="06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" Sep 05 02:20:56.942445 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.942429 2567 scope.go:117] "RemoveContainer" containerID="282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" Sep 05 02:20:56.946549 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946528 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-trusted-ca-bundle\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946625 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946564 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-tls-assets\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946625 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946607 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-0\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946743 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946637 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946743 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946668 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946743 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946709 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-1\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946743 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946737 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-tls\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946941 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946769 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-grpc-tls\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946941 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946804 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb9cr\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-kube-api-access-lb9cr\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946941 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946839 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-2\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946941 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946891 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-config-out\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.946941 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946916 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-config\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.947185 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946941 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-thanos-prometheus-http-client-file\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.947185 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946940 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:20:56.947185 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.946978 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-metrics-client-ca\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.947185 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.947007 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-web-config\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.947185 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.947038 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-metrics-client-certs\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.947185 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.947067 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-serving-certs-ca-bundle\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.947185 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.947093 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-db\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.947185 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.947122 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-kubelet-serving-ca-bundle\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.947185 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.947146 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-kube-rbac-proxy\") pod \"3cf09cdf-6abc-420e-b461-8474f191aa61\" (UID: \"3cf09cdf-6abc-420e-b461-8474f191aa61\") " Sep 05 02:20:56.947940 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.947385 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:56.947940 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.947549 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:20:56.948052 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.948009 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:20:56.948486 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.948461 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:20:56.949405 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.949379 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:20:56.950159 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.950050 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:20:56.950457 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.950429 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 02:20:56.950802 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.950744 2567 scope.go:117] "RemoveContainer" containerID="3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" Sep 05 02:20:56.950943 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.950916 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-config-out" (OuterVolumeSpecName: "config-out") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 02:20:56.951029 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.951006 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 02:20:56.951296 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.951273 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:20:56.951514 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.951448 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:20:56.951613 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.951567 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-kube-api-access-lb9cr" (OuterVolumeSpecName: "kube-api-access-lb9cr") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "kube-api-access-lb9cr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 02:20:56.951672 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.951626 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:20:56.951847 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.951821 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:20:56.952654 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.952620 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-config" (OuterVolumeSpecName: "config") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:20:56.952914 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.952864 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:20:56.952914 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.952891 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:20:56.953300 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.953280 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:20:56.953456 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.953435 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:20:56.958784 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.958762 2567 scope.go:117] "RemoveContainer" containerID="ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" Sep 05 02:20:56.963320 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.963296 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-web-config" (OuterVolumeSpecName: "web-config") pod "3cf09cdf-6abc-420e-b461-8474f191aa61" (UID: "3cf09cdf-6abc-420e-b461-8474f191aa61"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:20:56.966386 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.966370 2567 scope.go:117] "RemoveContainer" containerID="8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7" Sep 05 02:20:56.972705 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.972687 2567 scope.go:117] "RemoveContainer" containerID="0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" Sep 05 02:20:56.972982 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:20:56.972953 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": container with ID starting with 0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535 not found: ID does not exist" containerID="0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" Sep 05 02:20:56.973051 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.972996 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535"} err="failed to get container status \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": rpc error: code = NotFound desc = could not find container \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": container with ID starting with 0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535 not found: ID does not exist" Sep 05 02:20:56.973051 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.973041 2567 scope.go:117] "RemoveContainer" containerID="0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" Sep 05 02:20:56.973310 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:20:56.973294 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": container with ID starting with 0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41 not found: ID does not exist" containerID="0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" Sep 05 02:20:56.973360 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.973316 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41"} err="failed to get container status \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": rpc error: code = NotFound desc = could not find container \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": container with ID starting with 0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41 not found: ID does not exist" Sep 05 02:20:56.973360 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.973330 2567 scope.go:117] "RemoveContainer" containerID="06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" Sep 05 02:20:56.973572 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:20:56.973552 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": container with ID starting with 06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4 not found: ID does not exist" containerID="06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" Sep 05 02:20:56.973620 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.973581 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4"} err="failed to get container status \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": rpc error: code = NotFound desc = could not find container \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": container with ID starting with 06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4 not found: ID does not exist" Sep 05 02:20:56.973620 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.973601 2567 scope.go:117] "RemoveContainer" containerID="282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" Sep 05 02:20:56.973846 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:20:56.973830 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": container with ID starting with 282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569 not found: ID does not exist" containerID="282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" Sep 05 02:20:56.973914 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.973854 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569"} err="failed to get container status \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": rpc error: code = NotFound desc = could not find container \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": container with ID starting with 282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569 not found: ID does not exist" Sep 05 02:20:56.973914 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.973874 2567 scope.go:117] "RemoveContainer" containerID="3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" Sep 05 02:20:56.974154 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:20:56.974138 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": container with ID starting with 3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d not found: ID does not exist" containerID="3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" Sep 05 02:20:56.974191 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.974160 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d"} err="failed to get container status \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": rpc error: code = NotFound desc = could not find container \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": container with ID starting with 3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d not found: ID does not exist" Sep 05 02:20:56.974191 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.974173 2567 scope.go:117] "RemoveContainer" containerID="ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" Sep 05 02:20:56.974386 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:20:56.974370 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": container with ID starting with ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c not found: ID does not exist" containerID="ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" Sep 05 02:20:56.974427 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.974390 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c"} err="failed to get container status \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": rpc error: code = NotFound desc = could not find container \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": container with ID starting with ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c not found: ID does not exist" Sep 05 02:20:56.974427 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.974405 2567 scope.go:117] "RemoveContainer" containerID="8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7" Sep 05 02:20:56.974639 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:20:56.974621 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": container with ID starting with 8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7 not found: ID does not exist" containerID="8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7" Sep 05 02:20:56.974717 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.974641 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7"} err="failed to get container status \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": rpc error: code = NotFound desc = could not find container \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": container with ID starting with 8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7 not found: ID does not exist" Sep 05 02:20:56.974717 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.974653 2567 scope.go:117] "RemoveContainer" containerID="0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" Sep 05 02:20:56.974876 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.974858 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535"} err="failed to get container status \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": rpc error: code = NotFound desc = could not find container \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": container with ID starting with 0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535 not found: ID does not exist" Sep 05 02:20:56.974917 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.974877 2567 scope.go:117] "RemoveContainer" containerID="0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" Sep 05 02:20:56.975080 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975065 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41"} err="failed to get container status \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": rpc error: code = NotFound desc = could not find container \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": container with ID starting with 0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41 not found: ID does not exist" Sep 05 02:20:56.975126 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975080 2567 scope.go:117] "RemoveContainer" containerID="06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" Sep 05 02:20:56.975275 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975260 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4"} err="failed to get container status \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": rpc error: code = NotFound desc = could not find container \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": container with ID starting with 06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4 not found: ID does not exist" Sep 05 02:20:56.975321 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975275 2567 scope.go:117] "RemoveContainer" containerID="282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" Sep 05 02:20:56.975464 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975447 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569"} err="failed to get container status \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": rpc error: code = NotFound desc = could not find container \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": container with ID starting with 282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569 not found: ID does not exist" Sep 05 02:20:56.975524 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975464 2567 scope.go:117] "RemoveContainer" containerID="3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" Sep 05 02:20:56.975683 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975668 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d"} err="failed to get container status \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": rpc error: code = NotFound desc = could not find container \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": container with ID starting with 3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d not found: ID does not exist" Sep 05 02:20:56.975729 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975683 2567 scope.go:117] "RemoveContainer" containerID="ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" Sep 05 02:20:56.975888 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975871 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c"} err="failed to get container status \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": rpc error: code = NotFound desc = could not find container \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": container with ID starting with ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c not found: ID does not exist" Sep 05 02:20:56.975932 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.975888 2567 scope.go:117] "RemoveContainer" containerID="8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7" Sep 05 02:20:56.976097 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976082 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7"} err="failed to get container status \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": rpc error: code = NotFound desc = could not find container \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": container with ID starting with 8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7 not found: ID does not exist" Sep 05 02:20:56.976144 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976097 2567 scope.go:117] "RemoveContainer" containerID="0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" Sep 05 02:20:56.976270 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976251 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535"} err="failed to get container status \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": rpc error: code = NotFound desc = could not find container \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": container with ID starting with 0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535 not found: ID does not exist" Sep 05 02:20:56.976315 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976271 2567 scope.go:117] "RemoveContainer" containerID="0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" Sep 05 02:20:56.976535 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976517 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41"} err="failed to get container status \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": rpc error: code = NotFound desc = could not find container \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": container with ID starting with 0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41 not found: ID does not exist" Sep 05 02:20:56.976588 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976536 2567 scope.go:117] "RemoveContainer" containerID="06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" Sep 05 02:20:56.976747 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976729 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4"} err="failed to get container status \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": rpc error: code = NotFound desc = could not find container \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": container with ID starting with 06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4 not found: ID does not exist" Sep 05 02:20:56.976814 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976749 2567 scope.go:117] "RemoveContainer" containerID="282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" Sep 05 02:20:56.976990 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976973 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569"} err="failed to get container status \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": rpc error: code = NotFound desc = could not find container \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": container with ID starting with 282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569 not found: ID does not exist" Sep 05 02:20:56.977037 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.976992 2567 scope.go:117] "RemoveContainer" containerID="3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" Sep 05 02:20:56.977188 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.977171 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d"} err="failed to get container status \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": rpc error: code = NotFound desc = could not find container \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": container with ID starting with 3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d not found: ID does not exist" Sep 05 02:20:56.977250 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.977189 2567 scope.go:117] "RemoveContainer" containerID="ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" Sep 05 02:20:56.977397 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.977383 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c"} err="failed to get container status \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": rpc error: code = NotFound desc = could not find container \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": container with ID starting with ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c not found: ID does not exist" Sep 05 02:20:56.977441 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.977397 2567 scope.go:117] "RemoveContainer" containerID="8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7" Sep 05 02:20:56.977636 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.977615 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7"} err="failed to get container status \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": rpc error: code = NotFound desc = could not find container \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": container with ID starting with 8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7 not found: ID does not exist" Sep 05 02:20:56.977636 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.977633 2567 scope.go:117] "RemoveContainer" containerID="0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" Sep 05 02:20:56.977965 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.977935 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535"} err="failed to get container status \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": rpc error: code = NotFound desc = could not find container \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": container with ID starting with 0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535 not found: ID does not exist" Sep 05 02:20:56.977965 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.977959 2567 scope.go:117] "RemoveContainer" containerID="0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" Sep 05 02:20:56.978209 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.978189 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41"} err="failed to get container status \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": rpc error: code = NotFound desc = could not find container \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": container with ID starting with 0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41 not found: ID does not exist" Sep 05 02:20:56.978209 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.978208 2567 scope.go:117] "RemoveContainer" containerID="06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" Sep 05 02:20:56.978475 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.978456 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4"} err="failed to get container status \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": rpc error: code = NotFound desc = could not find container \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": container with ID starting with 06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4 not found: ID does not exist" Sep 05 02:20:56.978475 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.978475 2567 scope.go:117] "RemoveContainer" containerID="282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" Sep 05 02:20:56.978778 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.978762 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569"} err="failed to get container status \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": rpc error: code = NotFound desc = could not find container \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": container with ID starting with 282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569 not found: ID does not exist" Sep 05 02:20:56.978839 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.978782 2567 scope.go:117] "RemoveContainer" containerID="3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" Sep 05 02:20:56.979035 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979017 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d"} err="failed to get container status \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": rpc error: code = NotFound desc = could not find container \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": container with ID starting with 3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d not found: ID does not exist" Sep 05 02:20:56.979075 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979046 2567 scope.go:117] "RemoveContainer" containerID="ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" Sep 05 02:20:56.979294 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979278 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c"} err="failed to get container status \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": rpc error: code = NotFound desc = could not find container \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": container with ID starting with ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c not found: ID does not exist" Sep 05 02:20:56.979331 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979296 2567 scope.go:117] "RemoveContainer" containerID="8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7" Sep 05 02:20:56.979536 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979520 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7"} err="failed to get container status \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": rpc error: code = NotFound desc = could not find container \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": container with ID starting with 8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7 not found: ID does not exist" Sep 05 02:20:56.979580 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979537 2567 scope.go:117] "RemoveContainer" containerID="0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" Sep 05 02:20:56.979773 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979757 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535"} err="failed to get container status \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": rpc error: code = NotFound desc = could not find container \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": container with ID starting with 0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535 not found: ID does not exist" Sep 05 02:20:56.979812 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979773 2567 scope.go:117] "RemoveContainer" containerID="0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" Sep 05 02:20:56.979977 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979962 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41"} err="failed to get container status \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": rpc error: code = NotFound desc = could not find container \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": container with ID starting with 0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41 not found: ID does not exist" Sep 05 02:20:56.980013 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.979979 2567 scope.go:117] "RemoveContainer" containerID="06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" Sep 05 02:20:56.980191 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.980175 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4"} err="failed to get container status \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": rpc error: code = NotFound desc = could not find container \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": container with ID starting with 06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4 not found: ID does not exist" Sep 05 02:20:56.980231 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.980191 2567 scope.go:117] "RemoveContainer" containerID="282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" Sep 05 02:20:56.980431 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.980414 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569"} err="failed to get container status \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": rpc error: code = NotFound desc = could not find container \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": container with ID starting with 282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569 not found: ID does not exist" Sep 05 02:20:56.980487 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.980431 2567 scope.go:117] "RemoveContainer" containerID="3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" Sep 05 02:20:56.980673 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.980657 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d"} err="failed to get container status \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": rpc error: code = NotFound desc = could not find container \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": container with ID starting with 3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d not found: ID does not exist" Sep 05 02:20:56.980713 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.980674 2567 scope.go:117] "RemoveContainer" containerID="ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" Sep 05 02:20:56.980874 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.980856 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c"} err="failed to get container status \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": rpc error: code = NotFound desc = could not find container \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": container with ID starting with ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c not found: ID does not exist" Sep 05 02:20:56.980911 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.980874 2567 scope.go:117] "RemoveContainer" containerID="8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7" Sep 05 02:20:56.981060 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981045 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7"} err="failed to get container status \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": rpc error: code = NotFound desc = could not find container \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": container with ID starting with 8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7 not found: ID does not exist" Sep 05 02:20:56.981100 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981060 2567 scope.go:117] "RemoveContainer" containerID="0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535" Sep 05 02:20:56.981274 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981258 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535"} err="failed to get container status \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": rpc error: code = NotFound desc = could not find container \"0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535\": container with ID starting with 0a43f618a7b7e6a0ba48ccbe046d6c067d9e03627df983dcc020193fc00a5535 not found: ID does not exist" Sep 05 02:20:56.981274 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981274 2567 scope.go:117] "RemoveContainer" containerID="0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41" Sep 05 02:20:56.981454 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981440 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41"} err="failed to get container status \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": rpc error: code = NotFound desc = could not find container \"0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41\": container with ID starting with 0d095a0f321c4774064f1726603d6278eb87b9f523785e08806afc30bcf11e41 not found: ID does not exist" Sep 05 02:20:56.981512 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981454 2567 scope.go:117] "RemoveContainer" containerID="06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4" Sep 05 02:20:56.981685 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981669 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4"} err="failed to get container status \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": rpc error: code = NotFound desc = could not find container \"06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4\": container with ID starting with 06584346c779004d1ea2b5a718e69af0d5c9e21851eff6763af1f9f1c08753f4 not found: ID does not exist" Sep 05 02:20:56.981726 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981684 2567 scope.go:117] "RemoveContainer" containerID="282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569" Sep 05 02:20:56.981900 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981878 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569"} err="failed to get container status \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": rpc error: code = NotFound desc = could not find container \"282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569\": container with ID starting with 282743f1701fe0822602b1ce77b675000611ff2b11492738e95693b5fdbe7569 not found: ID does not exist" Sep 05 02:20:56.981950 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.981901 2567 scope.go:117] "RemoveContainer" containerID="3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d" Sep 05 02:20:56.982111 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.982095 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d"} err="failed to get container status \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": rpc error: code = NotFound desc = could not find container \"3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d\": container with ID starting with 3044e25ff52cafc90bf0cec45181d3169f0f78461e2aff1a051fb653d2ac647d not found: ID does not exist" Sep 05 02:20:56.982153 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.982114 2567 scope.go:117] "RemoveContainer" containerID="ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c" Sep 05 02:20:56.982303 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.982287 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c"} err="failed to get container status \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": rpc error: code = NotFound desc = could not find container \"ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c\": container with ID starting with ae4673b5632c20948e85bb5da7419ec3cfe9a4b0c97caf6b41e2c95ba2750d4c not found: ID does not exist" Sep 05 02:20:56.982303 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.982302 2567 scope.go:117] "RemoveContainer" containerID="8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7" Sep 05 02:20:56.982544 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:56.982523 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7"} err="failed to get container status \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": rpc error: code = NotFound desc = could not find container \"8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7\": container with ID starting with 8985d8faa4b9abedf73ac2f9f6f0ce1c0ceb3711faa17091bfae47f44f9b9ff7 not found: ID does not exist" Sep 05 02:20:57.047792 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047764 2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-grpc-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047792 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047791 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb9cr\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-kube-api-access-lb9cr\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047806 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047817 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-config-out\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047825 2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-config\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047834 2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047843 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-metrics-client-ca\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047851 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-web-config\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047859 2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-metrics-client-certs\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047870 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047884 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-db\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047893 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047901 2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-kube-rbac-proxy\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047909 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3cf09cdf-6abc-420e-b461-8474f191aa61-tls-assets\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047916 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047924 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047932 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047942 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3cf09cdf-6abc-420e-b461-8474f191aa61-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.047949 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.047952 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3cf09cdf-6abc-420e-b461-8474f191aa61-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:20:57.241019 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.240987 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 05 02:20:57.243950 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.243928 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 05 02:20:57.269076 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269051 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 05 02:20:57.269309 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269297 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="thanos-sidecar" Sep 05 02:20:57.269356 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269311 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="thanos-sidecar" Sep 05 02:20:57.269356 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269322 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="prometheus" Sep 05 02:20:57.269356 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269328 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="prometheus" Sep 05 02:20:57.269356 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269338 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="config-reloader" Sep 05 02:20:57.269356 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269344 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="config-reloader" Sep 05 02:20:57.269356 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269352 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="init-config-reloader" Sep 05 02:20:57.269356 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269357 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="init-config-reloader" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269362 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy-thanos" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269368 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy-thanos" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269373 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269379 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269388 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy-web" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269393 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy-web" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269439 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="prometheus" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269446 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="thanos-sidecar" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269452 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="config-reloader" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269457 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy-web" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269462 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy" Sep 05 02:20:57.269619 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.269466 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" containerName="kube-rbac-proxy-thanos" Sep 05 02:20:57.274885 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.274864 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.277768 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.277746 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 05 02:20:57.277909 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.277891 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 05 02:20:57.278149 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278128 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 05 02:20:57.278262 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278168 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-vvnr7c88qkli\"" Sep 05 02:20:57.278262 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278194 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 05 02:20:57.278262 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278194 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 05 02:20:57.278421 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278362 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 05 02:20:57.278873 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278857 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 05 02:20:57.278963 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278940 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 05 02:20:57.279019 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278977 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 05 02:20:57.279019 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278942 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 05 02:20:57.279085 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.278940 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7zppx\"" Sep 05 02:20:57.279085 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.279065 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 05 02:20:57.279200 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.279181 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 05 02:20:57.279283 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.279259 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 05 02:20:57.282403 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.282353 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 05 02:20:57.285833 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.285815 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 05 02:20:57.290098 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.290012 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 05 02:20:57.451246 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451208 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451425 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451263 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451425 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451330 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-config\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451425 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451357 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451425 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451386 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451425 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451408 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451432 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451463 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451487 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aceeb27b-700f-4df1-b20a-4b0e349a1878-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451543 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aceeb27b-700f-4df1-b20a-4b0e349a1878-config-out\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451595 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451653 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451682 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-web-config\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451719 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.451749 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451745 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.452083 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451770 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.452083 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451792 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.452083 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451845 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.452083 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.451874 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjb7c\" (UniqueName: \"kubernetes.io/projected/aceeb27b-700f-4df1-b20a-4b0e349a1878-kube-api-access-sjb7c\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553079 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.552991 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-config\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553079 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553031 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553079 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553050 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553079 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553073 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553369 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553099 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553369 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553124 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553369 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553198 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aceeb27b-700f-4df1-b20a-4b0e349a1878-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553369 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553242 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aceeb27b-700f-4df1-b20a-4b0e349a1878-config-out\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553369 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553369 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553296 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553369 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553325 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553369 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553355 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-web-config\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553384 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553406 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553434 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553460 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553523 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553547 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjb7c\" (UniqueName: \"kubernetes.io/projected/aceeb27b-700f-4df1-b20a-4b0e349a1878-kube-api-access-sjb7c\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553586 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553626 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.553796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553772 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.554244 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.553802 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.554579 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.554558 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.555024 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.555000 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.555449 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.555430 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.555747 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.555697 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.556807 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.556456 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aceeb27b-700f-4df1-b20a-4b0e349a1878-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.556807 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.556516 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.556967 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.556851 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.557288 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.557136 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.557373 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.557327 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.557800 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.557733 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-config\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.557911 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.557820 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aceeb27b-700f-4df1-b20a-4b0e349a1878-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.558257 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.558237 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.558413 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.558392 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aceeb27b-700f-4df1-b20a-4b0e349a1878-config-out\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.558767 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.558746 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.559027 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.559003 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.559383 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.559357 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-web-config\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.559659 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.559642 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aceeb27b-700f-4df1-b20a-4b0e349a1878-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.564700 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.564684 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjb7c\" (UniqueName: \"kubernetes.io/projected/aceeb27b-700f-4df1-b20a-4b0e349a1878-kube-api-access-sjb7c\") pod \"prometheus-k8s-0\" (UID: \"aceeb27b-700f-4df1-b20a-4b0e349a1878\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.585662 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.585639 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:20:57.866611 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.866580 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 05 02:20:57.867978 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:20:57.867944 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaceeb27b_700f_4df1_b20a_4b0e349a1878.slice/crio-c53fd23e0fc2777e727976b490084f2387651c2014240f1a6ede421ef3f67702 WatchSource:0}: Error finding container c53fd23e0fc2777e727976b490084f2387651c2014240f1a6ede421ef3f67702: Status 404 returned error can't find the container with id c53fd23e0fc2777e727976b490084f2387651c2014240f1a6ede421ef3f67702 Sep 05 02:20:57.926676 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:57.926642 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aceeb27b-700f-4df1-b20a-4b0e349a1878","Type":"ContainerStarted","Data":"c53fd23e0fc2777e727976b490084f2387651c2014240f1a6ede421ef3f67702"} Sep 05 02:20:58.934062 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:58.934021 2567 generic.go:356] "Generic (PLEG): container finished" podID="aceeb27b-700f-4df1-b20a-4b0e349a1878" containerID="d17fad87a4e25a387726c117a4238677ba401b763529de875b64f8d8d083223f" exitCode=0 Sep 05 02:20:58.934469 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:58.934068 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aceeb27b-700f-4df1-b20a-4b0e349a1878","Type":"ContainerDied","Data":"d17fad87a4e25a387726c117a4238677ba401b763529de875b64f8d8d083223f"} Sep 05 02:20:59.017838 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:59.017794 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf09cdf-6abc-420e-b461-8474f191aa61" path="/var/lib/kubelet/pods/3cf09cdf-6abc-420e-b461-8474f191aa61/volumes" Sep 05 02:20:59.944526 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:59.944470 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aceeb27b-700f-4df1-b20a-4b0e349a1878","Type":"ContainerStarted","Data":"8b1d4ef6381dcec85a73bd7c2522d1adf8414814a830c3ef31420de176853670"} Sep 05 02:20:59.944927 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:59.944562 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aceeb27b-700f-4df1-b20a-4b0e349a1878","Type":"ContainerStarted","Data":"c0f54d16143623965c68976c8affe59bd128ba3bea771b2f6dc824c72a8818a1"} Sep 05 02:20:59.944927 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:59.944599 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aceeb27b-700f-4df1-b20a-4b0e349a1878","Type":"ContainerStarted","Data":"d2d7722ad1cd2bf91921505038f7949bd715cadeccfc48478bfb3dd2205644fa"} Sep 05 02:20:59.944927 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:59.944625 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aceeb27b-700f-4df1-b20a-4b0e349a1878","Type":"ContainerStarted","Data":"f0d2f7a108920a698dd84b6287a0fdd766d360155442d5e47c915c6ac85c2193"} Sep 05 02:20:59.944927 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:59.944648 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aceeb27b-700f-4df1-b20a-4b0e349a1878","Type":"ContainerStarted","Data":"c23bd40ebc9fc60bc4436f7bff3b179414bed3aeb316cb60fb9639882ecbca43"} Sep 05 02:20:59.944927 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:59.944670 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aceeb27b-700f-4df1-b20a-4b0e349a1878","Type":"ContainerStarted","Data":"9cd75dfb7e983bb4ec7a1121ed062a7f1fa261027b609a71decab8da231c80f4"} Sep 05 02:20:59.979661 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:20:59.979522 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.9612242479999997 podStartE2EDuration="2.979473055s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.248807ms" isStatefulPod=false podCreationTimestamp="2026-09-05 02:20:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:20:59.978212759 +0000 UTC m=+183.464071734" watchObservedRunningTime="2026-09-05 02:20:59.979473055 +0000 UTC m=+183.465332033" Sep 05 02:21:02.586616 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:21:02.586581 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:21:57.586708 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:21:57.586668 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:21:57.602640 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:21:57.602614 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:21:58.179847 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:21:58.179819 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 05 02:22:56.919278 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:22:56.919245 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:22:56.922074 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:22:56.922050 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:22:56.930401 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:22:56.930380 2567 kubelet.go:1648] "Image garbage collection succeeded" Sep 05 02:27:56.943896 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:27:56.943872 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:27:56.945806 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:27:56.945780 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:30:58.486089 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.486055 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj"] Sep 05 02:30:58.489329 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.489307 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:30:58.491675 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.491656 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-a6d98-serving-cert\"" Sep 05 02:30:58.491802 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.491759 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-a6d98-kube-rbac-proxy-sar-config\"" Sep 05 02:30:58.491907 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.491891 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 05 02:30:58.492579 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.492564 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7f65m\"" Sep 05 02:30:58.496283 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.496263 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj"] Sep 05 02:30:58.550325 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.550301 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec85ca3-923d-49bc-8624-8027870b9244-proxy-tls\") pod \"switch-graph-a6d98-665c6779cc-k6wxj\" (UID: \"dec85ca3-923d-49bc-8624-8027870b9244\") " pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:30:58.550439 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.550335 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec85ca3-923d-49bc-8624-8027870b9244-openshift-service-ca-bundle\") pod \"switch-graph-a6d98-665c6779cc-k6wxj\" (UID: \"dec85ca3-923d-49bc-8624-8027870b9244\") " pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:30:58.650975 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.650945 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec85ca3-923d-49bc-8624-8027870b9244-proxy-tls\") pod \"switch-graph-a6d98-665c6779cc-k6wxj\" (UID: \"dec85ca3-923d-49bc-8624-8027870b9244\") " pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:30:58.651107 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.650985 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec85ca3-923d-49bc-8624-8027870b9244-openshift-service-ca-bundle\") pod \"switch-graph-a6d98-665c6779cc-k6wxj\" (UID: \"dec85ca3-923d-49bc-8624-8027870b9244\") " pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:30:58.651568 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.651553 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec85ca3-923d-49bc-8624-8027870b9244-openshift-service-ca-bundle\") pod \"switch-graph-a6d98-665c6779cc-k6wxj\" (UID: \"dec85ca3-923d-49bc-8624-8027870b9244\") " pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:30:58.653281 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.653266 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec85ca3-923d-49bc-8624-8027870b9244-proxy-tls\") pod \"switch-graph-a6d98-665c6779cc-k6wxj\" (UID: \"dec85ca3-923d-49bc-8624-8027870b9244\") " pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:30:58.799350 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:58.799297 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:30:59.069483 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:59.069417 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj"] Sep 05 02:30:59.072131 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:30:59.072106 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddec85ca3_923d_49bc_8624_8027870b9244.slice/crio-e9bccafc184a79f27124ae278ecba377ae2610d5d35f604fd0820e26bc570b0d WatchSource:0}: Error finding container e9bccafc184a79f27124ae278ecba377ae2610d5d35f604fd0820e26bc570b0d: Status 404 returned error can't find the container with id e9bccafc184a79f27124ae278ecba377ae2610d5d35f604fd0820e26bc570b0d Sep 05 02:30:59.073940 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:59.073925 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 05 02:30:59.157462 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:30:59.157429 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" event={"ID":"dec85ca3-923d-49bc-8624-8027870b9244","Type":"ContainerStarted","Data":"e9bccafc184a79f27124ae278ecba377ae2610d5d35f604fd0820e26bc570b0d"} Sep 05 02:31:02.170777 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:02.170703 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" event={"ID":"dec85ca3-923d-49bc-8624-8027870b9244","Type":"ContainerStarted","Data":"31d508171a5fd82b321ffc67b97db46c27820b4cfef5eb924f3321dc41b37866"} Sep 05 02:31:02.171084 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:02.170884 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:31:08.179820 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:08.179791 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:31:08.196461 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:08.196415 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" podStartSLOduration=7.374077632 podStartE2EDuration="10.196396388s" totalImagesPullingTime="2.822318756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:31:02.187289338 +0000 UTC m=+785.673148310" watchObservedRunningTime="2026-09-05 02:31:08.196396388 +0000 UTC m=+791.682255361" Sep 05 02:31:08.694968 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:08.694934 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj"] Sep 05 02:31:08.695130 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:08.695111 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" containerID="cri-o://31d508171a5fd82b321ffc67b97db46c27820b4cfef5eb924f3321dc41b37866" gracePeriod=30 Sep 05 02:31:13.179075 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:13.179026 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:31:18.178515 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:18.178466 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:31:23.179157 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:23.179117 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:31:23.179686 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:23.179260 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:31:28.179640 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:28.179603 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:31:33.179967 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:33.179905 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:31:38.179526 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:38.179473 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:31:39.308249 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:39.308212 2567 generic.go:356] "Generic (PLEG): container finished" podID="dec85ca3-923d-49bc-8624-8027870b9244" containerID="31d508171a5fd82b321ffc67b97db46c27820b4cfef5eb924f3321dc41b37866" exitCode=0 Sep 05 02:31:39.308583 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:39.308263 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" event={"ID":"dec85ca3-923d-49bc-8624-8027870b9244","Type":"ContainerDied","Data":"31d508171a5fd82b321ffc67b97db46c27820b4cfef5eb924f3321dc41b37866"} Sep 05 02:31:39.343587 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:39.343567 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:31:39.485472 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:39.485396 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec85ca3-923d-49bc-8624-8027870b9244-proxy-tls\") pod \"dec85ca3-923d-49bc-8624-8027870b9244\" (UID: \"dec85ca3-923d-49bc-8624-8027870b9244\") " Sep 05 02:31:39.485472 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:39.485434 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec85ca3-923d-49bc-8624-8027870b9244-openshift-service-ca-bundle\") pod \"dec85ca3-923d-49bc-8624-8027870b9244\" (UID: \"dec85ca3-923d-49bc-8624-8027870b9244\") " Sep 05 02:31:39.485901 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:39.485850 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dec85ca3-923d-49bc-8624-8027870b9244-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "dec85ca3-923d-49bc-8624-8027870b9244" (UID: "dec85ca3-923d-49bc-8624-8027870b9244"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:31:39.487708 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:39.487681 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dec85ca3-923d-49bc-8624-8027870b9244-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dec85ca3-923d-49bc-8624-8027870b9244" (UID: "dec85ca3-923d-49bc-8624-8027870b9244"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:31:39.586239 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:39.586205 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec85ca3-923d-49bc-8624-8027870b9244-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:31:39.586239 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:39.586232 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec85ca3-923d-49bc-8624-8027870b9244-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:31:40.313576 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:40.313544 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" event={"ID":"dec85ca3-923d-49bc-8624-8027870b9244","Type":"ContainerDied","Data":"e9bccafc184a79f27124ae278ecba377ae2610d5d35f604fd0820e26bc570b0d"} Sep 05 02:31:40.313576 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:40.313573 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj" Sep 05 02:31:40.314070 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:40.313600 2567 scope.go:117] "RemoveContainer" containerID="31d508171a5fd82b321ffc67b97db46c27820b4cfef5eb924f3321dc41b37866" Sep 05 02:31:40.334114 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:40.334092 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj"] Sep 05 02:31:40.337311 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:40.337293 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-a6d98-665c6779cc-k6wxj"] Sep 05 02:31:41.016998 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:41.016966 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dec85ca3-923d-49bc-8624-8027870b9244" path="/var/lib/kubelet/pods/dec85ca3-923d-49bc-8624-8027870b9244/volumes" Sep 05 02:31:48.463760 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.463726 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk"] Sep 05 02:31:48.464113 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.464021 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" Sep 05 02:31:48.464113 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.464032 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" Sep 05 02:31:48.464113 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.464089 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dec85ca3-923d-49bc-8624-8027870b9244" containerName="switch-graph-a6d98" Sep 05 02:31:48.468476 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.468461 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:48.471084 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.471066 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 05 02:31:48.471209 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.471068 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Sep 05 02:31:48.471209 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.471066 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Sep 05 02:31:48.471209 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.471140 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7f65m\"" Sep 05 02:31:48.479155 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.479133 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk"] Sep 05 02:31:48.558722 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.558701 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb4131e5-5b53-46bd-a97e-9680cff01c9f-proxy-tls\") pod \"model-chainer-fffbcdb68-98rmk\" (UID: \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\") " pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:48.558827 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.558767 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb4131e5-5b53-46bd-a97e-9680cff01c9f-openshift-service-ca-bundle\") pod \"model-chainer-fffbcdb68-98rmk\" (UID: \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\") " pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:48.659286 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.659267 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb4131e5-5b53-46bd-a97e-9680cff01c9f-openshift-service-ca-bundle\") pod \"model-chainer-fffbcdb68-98rmk\" (UID: \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\") " pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:48.659342 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.659309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb4131e5-5b53-46bd-a97e-9680cff01c9f-proxy-tls\") pod \"model-chainer-fffbcdb68-98rmk\" (UID: \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\") " pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:48.659857 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.659837 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb4131e5-5b53-46bd-a97e-9680cff01c9f-openshift-service-ca-bundle\") pod \"model-chainer-fffbcdb68-98rmk\" (UID: \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\") " pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:48.661802 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.661777 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb4131e5-5b53-46bd-a97e-9680cff01c9f-proxy-tls\") pod \"model-chainer-fffbcdb68-98rmk\" (UID: \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\") " pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:48.778730 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:48.778634 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:49.048643 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:49.048609 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk"] Sep 05 02:31:49.051154 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:31:49.051119 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb4131e5_5b53_46bd_a97e_9680cff01c9f.slice/crio-1ef304d4e1e1a0d6f63982256c573406ea83fac54f13f5c29bb32261912ac03b WatchSource:0}: Error finding container 1ef304d4e1e1a0d6f63982256c573406ea83fac54f13f5c29bb32261912ac03b: Status 404 returned error can't find the container with id 1ef304d4e1e1a0d6f63982256c573406ea83fac54f13f5c29bb32261912ac03b Sep 05 02:31:49.351118 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:49.351031 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" event={"ID":"bb4131e5-5b53-46bd-a97e-9680cff01c9f","Type":"ContainerStarted","Data":"95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b"} Sep 05 02:31:49.351118 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:49.351080 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" event={"ID":"bb4131e5-5b53-46bd-a97e-9680cff01c9f","Type":"ContainerStarted","Data":"1ef304d4e1e1a0d6f63982256c573406ea83fac54f13f5c29bb32261912ac03b"} Sep 05 02:31:49.351308 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:49.351169 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:49.367057 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:49.367012 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" podStartSLOduration=1.366997957 podStartE2EDuration="1.366997957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:31:49.365277059 +0000 UTC m=+832.851136032" watchObservedRunningTime="2026-09-05 02:31:49.366997957 +0000 UTC m=+832.852856930" Sep 05 02:31:55.360893 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:55.360865 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:31:58.558836 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:58.558804 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk"] Sep 05 02:31:58.559246 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:31:58.559004 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" containerID="cri-o://95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b" gracePeriod=30 Sep 05 02:32:00.359312 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:00.359274 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:32:05.358795 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:05.358756 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:32:08.920143 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:08.920112 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4"] Sep 05 02:32:08.924813 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:08.924793 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:08.927870 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:08.927845 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-3f635-kube-rbac-proxy-sar-config\"" Sep 05 02:32:08.927989 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:08.927890 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-3f635-serving-cert\"" Sep 05 02:32:08.931210 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:08.931185 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4"] Sep 05 02:32:08.992571 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:08.992529 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da64be77-28d0-4d1f-a404-fe6abafdaf0c-proxy-tls\") pod \"switch-graph-3f635-7c445694c5-k6nq4\" (UID: \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\") " pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:08.992748 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:08.992596 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da64be77-28d0-4d1f-a404-fe6abafdaf0c-openshift-service-ca-bundle\") pod \"switch-graph-3f635-7c445694c5-k6nq4\" (UID: \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\") " pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:09.093620 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:09.093580 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da64be77-28d0-4d1f-a404-fe6abafdaf0c-openshift-service-ca-bundle\") pod \"switch-graph-3f635-7c445694c5-k6nq4\" (UID: \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\") " pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:09.093796 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:09.093651 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da64be77-28d0-4d1f-a404-fe6abafdaf0c-proxy-tls\") pod \"switch-graph-3f635-7c445694c5-k6nq4\" (UID: \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\") " pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:09.094381 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:09.094359 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da64be77-28d0-4d1f-a404-fe6abafdaf0c-openshift-service-ca-bundle\") pod \"switch-graph-3f635-7c445694c5-k6nq4\" (UID: \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\") " pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:09.096217 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:09.096194 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da64be77-28d0-4d1f-a404-fe6abafdaf0c-proxy-tls\") pod \"switch-graph-3f635-7c445694c5-k6nq4\" (UID: \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\") " pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:09.235648 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:09.235621 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:09.512038 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:09.511876 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4"] Sep 05 02:32:09.514940 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:32:09.514912 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda64be77_28d0_4d1f_a404_fe6abafdaf0c.slice/crio-eae3ed78d5be84217b30f4c9d4ba656bf780ab2e03ee8580e9f74d9db3697e05 WatchSource:0}: Error finding container eae3ed78d5be84217b30f4c9d4ba656bf780ab2e03ee8580e9f74d9db3697e05: Status 404 returned error can't find the container with id eae3ed78d5be84217b30f4c9d4ba656bf780ab2e03ee8580e9f74d9db3697e05 Sep 05 02:32:10.358303 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:10.358267 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:32:10.358700 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:10.358364 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:32:10.431092 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:10.431057 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" event={"ID":"da64be77-28d0-4d1f-a404-fe6abafdaf0c","Type":"ContainerStarted","Data":"a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a"} Sep 05 02:32:10.431265 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:10.431112 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" event={"ID":"da64be77-28d0-4d1f-a404-fe6abafdaf0c","Type":"ContainerStarted","Data":"eae3ed78d5be84217b30f4c9d4ba656bf780ab2e03ee8580e9f74d9db3697e05"} Sep 05 02:32:10.431265 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:10.431155 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:10.447276 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:10.447231 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" podStartSLOduration=2.447215221 podStartE2EDuration="2.447215221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:32:10.446247321 +0000 UTC m=+853.932106308" watchObservedRunningTime="2026-09-05 02:32:10.447215221 +0000 UTC m=+853.933074195" Sep 05 02:32:15.358859 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:15.358791 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:32:16.440785 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:16.440752 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:32:20.359112 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:20.359071 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:32:25.359167 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:25.359121 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:32:28.706085 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:28.706061 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:32:28.766447 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:28.766415 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb4131e5-5b53-46bd-a97e-9680cff01c9f-proxy-tls\") pod \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\" (UID: \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\") " Sep 05 02:32:28.766608 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:28.766561 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb4131e5-5b53-46bd-a97e-9680cff01c9f-openshift-service-ca-bundle\") pod \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\" (UID: \"bb4131e5-5b53-46bd-a97e-9680cff01c9f\") " Sep 05 02:32:28.766882 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:28.766858 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb4131e5-5b53-46bd-a97e-9680cff01c9f-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "bb4131e5-5b53-46bd-a97e-9680cff01c9f" (UID: "bb4131e5-5b53-46bd-a97e-9680cff01c9f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:32:28.768710 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:28.768693 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb4131e5-5b53-46bd-a97e-9680cff01c9f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bb4131e5-5b53-46bd-a97e-9680cff01c9f" (UID: "bb4131e5-5b53-46bd-a97e-9680cff01c9f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:32:28.867518 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:28.867429 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb4131e5-5b53-46bd-a97e-9680cff01c9f-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:32:28.867518 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:28.867453 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb4131e5-5b53-46bd-a97e-9680cff01c9f-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:32:29.511618 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:29.511583 2567 generic.go:356] "Generic (PLEG): container finished" podID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerID="95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b" exitCode=0 Sep 05 02:32:29.511800 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:29.511650 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" Sep 05 02:32:29.511800 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:29.511675 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" event={"ID":"bb4131e5-5b53-46bd-a97e-9680cff01c9f","Type":"ContainerDied","Data":"95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b"} Sep 05 02:32:29.511800 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:29.511751 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk" event={"ID":"bb4131e5-5b53-46bd-a97e-9680cff01c9f","Type":"ContainerDied","Data":"1ef304d4e1e1a0d6f63982256c573406ea83fac54f13f5c29bb32261912ac03b"} Sep 05 02:32:29.511800 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:29.511773 2567 scope.go:117] "RemoveContainer" containerID="95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b" Sep 05 02:32:29.520012 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:29.519995 2567 scope.go:117] "RemoveContainer" containerID="95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b" Sep 05 02:32:29.520283 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:32:29.520264 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b\": container with ID starting with 95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b not found: ID does not exist" containerID="95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b" Sep 05 02:32:29.520326 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:29.520291 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b"} err="failed to get container status \"95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b\": rpc error: code = NotFound desc = could not find container \"95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b\": container with ID starting with 95a7b8cfb1d02c1c1296dea3fde8a50d3c0f4f560ea9f93daee92eead4b97d2b not found: ID does not exist" Sep 05 02:32:29.528622 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:29.528599 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk"] Sep 05 02:32:29.532255 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:29.532234 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-fffbcdb68-98rmk"] Sep 05 02:32:31.017781 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:31.017751 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" path="/var/lib/kubelet/pods/bb4131e5-5b53-46bd-a97e-9680cff01c9f/volumes" Sep 05 02:32:56.970081 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:56.970059 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:32:56.971351 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:56.971332 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:32:58.762641 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.762607 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv"] Sep 05 02:32:58.763006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.762915 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" Sep 05 02:32:58.763006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.762926 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" Sep 05 02:32:58.763006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.762980 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb4131e5-5b53-46bd-a97e-9680cff01c9f" containerName="model-chainer" Sep 05 02:32:58.766547 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.766532 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:32:58.768549 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.768528 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-9f825-kube-rbac-proxy-sar-config\"" Sep 05 02:32:58.768678 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.768550 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-9f825-serving-cert\"" Sep 05 02:32:58.775197 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.775175 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv"] Sep 05 02:32:58.815209 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.815181 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-openshift-service-ca-bundle\") pod \"sequence-graph-9f825-7798c68c55-glwtv\" (UID: \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\") " pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:32:58.815348 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.815214 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-proxy-tls\") pod \"sequence-graph-9f825-7798c68c55-glwtv\" (UID: \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\") " pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:32:58.916299 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.916264 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-openshift-service-ca-bundle\") pod \"sequence-graph-9f825-7798c68c55-glwtv\" (UID: \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\") " pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:32:58.916299 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.916296 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-proxy-tls\") pod \"sequence-graph-9f825-7798c68c55-glwtv\" (UID: \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\") " pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:32:58.916539 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:32:58.916407 2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-9f825-serving-cert: secret "sequence-graph-9f825-serving-cert" not found Sep 05 02:32:58.916539 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:32:58.916460 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-proxy-tls podName:2ad50fb5-d22e-4ef2-9e67-bb75a48f958c nodeName:}" failed. No retries permitted until 2026-09-05 02:32:59.416443833 +0000 UTC m=+902.902302797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-proxy-tls") pod "sequence-graph-9f825-7798c68c55-glwtv" (UID: "2ad50fb5-d22e-4ef2-9e67-bb75a48f958c") : secret "sequence-graph-9f825-serving-cert" not found Sep 05 02:32:58.916884 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:58.916864 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-openshift-service-ca-bundle\") pod \"sequence-graph-9f825-7798c68c55-glwtv\" (UID: \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\") " pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:32:59.420920 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:59.420879 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-proxy-tls\") pod \"sequence-graph-9f825-7798c68c55-glwtv\" (UID: \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\") " pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:32:59.423501 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:59.423469 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-proxy-tls\") pod \"sequence-graph-9f825-7798c68c55-glwtv\" (UID: \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\") " pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:32:59.677776 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:59.677683 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:32:59.951427 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:32:59.951400 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv"] Sep 05 02:32:59.953970 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:32:59.953929 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad50fb5_d22e_4ef2_9e67_bb75a48f958c.slice/crio-3b88c653ead1923dbcc19e8f4f157b2ab9e4006ccca9d2daf5dbe40b2eddeb80 WatchSource:0}: Error finding container 3b88c653ead1923dbcc19e8f4f157b2ab9e4006ccca9d2daf5dbe40b2eddeb80: Status 404 returned error can't find the container with id 3b88c653ead1923dbcc19e8f4f157b2ab9e4006ccca9d2daf5dbe40b2eddeb80 Sep 05 02:33:00.630933 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:33:00.630892 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" event={"ID":"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c","Type":"ContainerStarted","Data":"6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5"} Sep 05 02:33:00.631103 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:33:00.630942 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" event={"ID":"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c","Type":"ContainerStarted","Data":"3b88c653ead1923dbcc19e8f4f157b2ab9e4006ccca9d2daf5dbe40b2eddeb80"} Sep 05 02:33:00.631103 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:33:00.631021 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:33:00.647569 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:33:00.647521 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" podStartSLOduration=2.647487102 podStartE2EDuration="2.647487102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:32:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:33:00.645915714 +0000 UTC m=+904.131774691" watchObservedRunningTime="2026-09-05 02:33:00.647487102 +0000 UTC m=+904.133346106" Sep 05 02:33:06.640569 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:33:06.640542 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:37:56.995176 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:37:56.995146 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:37:56.997096 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:37:56.997071 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:40:23.568553 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:23.568519 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4"] Sep 05 02:40:23.569110 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:23.568729 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" containerID="cri-o://a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a" gracePeriod=30 Sep 05 02:40:26.439070 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:26.439031 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:40:31.439109 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:31.439069 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:40:36.440082 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:36.440039 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:40:36.440529 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:36.440147 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:40:41.439628 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:41.439588 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:40:46.439467 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:46.439429 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:40:51.439768 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:51.439727 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:40:54.204772 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.204751 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:40:54.220661 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.220636 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da64be77-28d0-4d1f-a404-fe6abafdaf0c-openshift-service-ca-bundle\") pod \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\" (UID: \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\") " Sep 05 02:40:54.220766 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.220666 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da64be77-28d0-4d1f-a404-fe6abafdaf0c-proxy-tls\") pod \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\" (UID: \"da64be77-28d0-4d1f-a404-fe6abafdaf0c\") " Sep 05 02:40:54.220981 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.220955 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da64be77-28d0-4d1f-a404-fe6abafdaf0c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "da64be77-28d0-4d1f-a404-fe6abafdaf0c" (UID: "da64be77-28d0-4d1f-a404-fe6abafdaf0c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:40:54.223305 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.223283 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da64be77-28d0-4d1f-a404-fe6abafdaf0c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "da64be77-28d0-4d1f-a404-fe6abafdaf0c" (UID: "da64be77-28d0-4d1f-a404-fe6abafdaf0c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:40:54.321731 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.321696 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da64be77-28d0-4d1f-a404-fe6abafdaf0c-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:40:54.321731 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.321724 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da64be77-28d0-4d1f-a404-fe6abafdaf0c-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:40:54.449088 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.449016 2567 generic.go:356] "Generic (PLEG): container finished" podID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerID="a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a" exitCode=0 Sep 05 02:40:54.449088 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.449071 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" event={"ID":"da64be77-28d0-4d1f-a404-fe6abafdaf0c","Type":"ContainerDied","Data":"a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a"} Sep 05 02:40:54.449251 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.449114 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" event={"ID":"da64be77-28d0-4d1f-a404-fe6abafdaf0c","Type":"ContainerDied","Data":"eae3ed78d5be84217b30f4c9d4ba656bf780ab2e03ee8580e9f74d9db3697e05"} Sep 05 02:40:54.449251 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.449119 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4" Sep 05 02:40:54.449251 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.449135 2567 scope.go:117] "RemoveContainer" containerID="a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a" Sep 05 02:40:54.457860 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.457841 2567 scope.go:117] "RemoveContainer" containerID="a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a" Sep 05 02:40:54.458132 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:40:54.458115 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a\": container with ID starting with a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a not found: ID does not exist" containerID="a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a" Sep 05 02:40:54.458178 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.458141 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a"} err="failed to get container status \"a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a\": rpc error: code = NotFound desc = could not find container \"a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a\": container with ID starting with a8d19d9b7d4c495f5f7b62d03e07b3fc40b97e0c02ce038f8e8d358bb3f08d2a not found: ID does not exist" Sep 05 02:40:54.469802 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.469782 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4"] Sep 05 02:40:54.474020 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:54.473999 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3f635-7c445694c5-k6nq4"] Sep 05 02:40:55.017731 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:40:55.017696 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" path="/var/lib/kubelet/pods/da64be77-28d0-4d1f-a404-fe6abafdaf0c/volumes" Sep 05 02:41:13.479065 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:13.479021 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv"] Sep 05 02:41:13.479605 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:13.479219 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" containerID="cri-o://6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5" gracePeriod=30 Sep 05 02:41:16.638544 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:16.638489 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:21.638853 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:21.638814 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:23.795165 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.795134 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk"] Sep 05 02:41:23.795539 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.795443 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" Sep 05 02:41:23.795539 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.795454 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" Sep 05 02:41:23.795539 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.795523 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="da64be77-28d0-4d1f-a404-fe6abafdaf0c" containerName="switch-graph-3f635" Sep 05 02:41:23.798616 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.798601 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:23.800733 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.800713 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-12d88-serving-cert\"" Sep 05 02:41:23.800836 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.800814 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-12d88-kube-rbac-proxy-sar-config\"" Sep 05 02:41:23.807404 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.807379 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk"] Sep 05 02:41:23.843863 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.843840 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-openshift-service-ca-bundle\") pod \"ensemble-graph-12d88-7bbf487fc6-js8jk\" (UID: \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\") " pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:23.843990 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.843872 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-proxy-tls\") pod \"ensemble-graph-12d88-7bbf487fc6-js8jk\" (UID: \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\") " pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:23.944646 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.944620 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-proxy-tls\") pod \"ensemble-graph-12d88-7bbf487fc6-js8jk\" (UID: \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\") " pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:23.944763 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.944688 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-openshift-service-ca-bundle\") pod \"ensemble-graph-12d88-7bbf487fc6-js8jk\" (UID: \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\") " pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:23.945218 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.945197 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-openshift-service-ca-bundle\") pod \"ensemble-graph-12d88-7bbf487fc6-js8jk\" (UID: \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\") " pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:23.947071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:23.947055 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-proxy-tls\") pod \"ensemble-graph-12d88-7bbf487fc6-js8jk\" (UID: \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\") " pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:24.109059 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:24.108967 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:24.375457 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:24.375381 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk"] Sep 05 02:41:24.378952 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:41:24.378928 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb67bd79a_2dec_4e98_8fa2_f6c4b5cd07a0.slice/crio-b935fed8fcedc405b81bcbc3b554067e2d355e6740ec4e360d780455f2d4e974 WatchSource:0}: Error finding container b935fed8fcedc405b81bcbc3b554067e2d355e6740ec4e360d780455f2d4e974: Status 404 returned error can't find the container with id b935fed8fcedc405b81bcbc3b554067e2d355e6740ec4e360d780455f2d4e974 Sep 05 02:41:24.567379 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:24.567345 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" event={"ID":"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0","Type":"ContainerStarted","Data":"b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd"} Sep 05 02:41:24.567559 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:24.567401 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" event={"ID":"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0","Type":"ContainerStarted","Data":"b935fed8fcedc405b81bcbc3b554067e2d355e6740ec4e360d780455f2d4e974"} Sep 05 02:41:24.567559 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:24.567432 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:24.583166 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:24.583127 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" podStartSLOduration=1.5831129339999999 podStartE2EDuration="1.583112934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:41:24.582727077 +0000 UTC m=+1408.068586055" watchObservedRunningTime="2026-09-05 02:41:24.583112934 +0000 UTC m=+1408.068971907" Sep 05 02:41:26.638439 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:26.638399 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:26.638832 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:26.638514 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:41:30.576912 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:30.576881 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:31.639124 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:31.639088 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:33.869787 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:33.869709 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk"] Sep 05 02:41:33.870165 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:33.869924 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" containerID="cri-o://b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd" gracePeriod=30 Sep 05 02:41:35.575225 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:35.575190 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:36.638976 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:36.638942 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:40.574979 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:40.574933 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:41.638816 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:41.638780 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:43.621336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.621307 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:41:43.641299 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.641272 2567 generic.go:356] "Generic (PLEG): container finished" podID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerID="6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5" exitCode=0 Sep 05 02:41:43.641420 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.641318 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" Sep 05 02:41:43.641420 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.641322 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" event={"ID":"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c","Type":"ContainerDied","Data":"6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5"} Sep 05 02:41:43.641514 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.641444 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv" event={"ID":"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c","Type":"ContainerDied","Data":"3b88c653ead1923dbcc19e8f4f157b2ab9e4006ccca9d2daf5dbe40b2eddeb80"} Sep 05 02:41:43.641514 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.641479 2567 scope.go:117] "RemoveContainer" containerID="6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5" Sep 05 02:41:43.649436 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.649409 2567 scope.go:117] "RemoveContainer" containerID="6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5" Sep 05 02:41:43.649769 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:41:43.649752 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5\": container with ID starting with 6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5 not found: ID does not exist" containerID="6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5" Sep 05 02:41:43.649836 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.649777 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5"} err="failed to get container status \"6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5\": rpc error: code = NotFound desc = could not find container \"6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5\": container with ID starting with 6ebe941989ef90d4caa38b08c8b9e578b7c80dcab2b641a6027bbc30056c78e5 not found: ID does not exist" Sep 05 02:41:43.714039 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.714015 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-openshift-service-ca-bundle\") pod \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\" (UID: \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\") " Sep 05 02:41:43.714170 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.714074 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-proxy-tls\") pod \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\" (UID: \"2ad50fb5-d22e-4ef2-9e67-bb75a48f958c\") " Sep 05 02:41:43.714415 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.714397 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" (UID: "2ad50fb5-d22e-4ef2-9e67-bb75a48f958c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:41:43.716265 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.716245 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" (UID: "2ad50fb5-d22e-4ef2-9e67-bb75a48f958c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:41:43.815283 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.815188 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:41:43.815283 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.815233 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:41:43.961973 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.961944 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv"] Sep 05 02:41:43.965384 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:43.965362 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9f825-7798c68c55-glwtv"] Sep 05 02:41:45.016845 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:45.016809 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" path="/var/lib/kubelet/pods/2ad50fb5-d22e-4ef2-9e67-bb75a48f958c/volumes" Sep 05 02:41:45.575026 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:45.574993 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:45.575209 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:45.575087 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:41:50.575781 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:50.575700 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:41:55.575867 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:41:55.575831 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:42:00.575030 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:00.574987 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:42:04.060096 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.060072 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:42:04.069394 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.069375 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-proxy-tls\") pod \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\" (UID: \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\") " Sep 05 02:42:04.069481 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.069445 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-openshift-service-ca-bundle\") pod \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\" (UID: \"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0\") " Sep 05 02:42:04.069770 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.069749 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" (UID: "b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:42:04.071449 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.071431 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" (UID: "b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:42:04.170632 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.170571 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:42:04.170632 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.170595 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:42:04.723084 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.723049 2567 generic.go:356] "Generic (PLEG): container finished" podID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerID="b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd" exitCode=0 Sep 05 02:42:04.723272 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.723092 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" event={"ID":"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0","Type":"ContainerDied","Data":"b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd"} Sep 05 02:42:04.723272 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.723096 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" Sep 05 02:42:04.723272 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.723143 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk" event={"ID":"b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0","Type":"ContainerDied","Data":"b935fed8fcedc405b81bcbc3b554067e2d355e6740ec4e360d780455f2d4e974"} Sep 05 02:42:04.723272 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.723171 2567 scope.go:117] "RemoveContainer" containerID="b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd" Sep 05 02:42:04.731524 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.731487 2567 scope.go:117] "RemoveContainer" containerID="b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd" Sep 05 02:42:04.731797 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:42:04.731779 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd\": container with ID starting with b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd not found: ID does not exist" containerID="b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd" Sep 05 02:42:04.731858 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.731806 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd"} err="failed to get container status \"b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd\": rpc error: code = NotFound desc = could not find container \"b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd\": container with ID starting with b710d4496473639ffdeb656c62d51d42fe124d68e622e621924a6710ba4646cd not found: ID does not exist" Sep 05 02:42:04.742326 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.742304 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk"] Sep 05 02:42:04.745528 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:04.745510 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-12d88-7bbf487fc6-js8jk"] Sep 05 02:42:05.016814 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:05.016744 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" path="/var/lib/kubelet/pods/b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0/volumes" Sep 05 02:42:13.678210 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.678174 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt"] Sep 05 02:42:13.678583 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.678473 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" Sep 05 02:42:13.678583 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.678484 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" Sep 05 02:42:13.678583 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.678509 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" Sep 05 02:42:13.678583 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.678514 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" Sep 05 02:42:13.678583 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.678565 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67bd79a-2dec-4e98-8fa2-f6c4b5cd07a0" containerName="ensemble-graph-12d88" Sep 05 02:42:13.678583 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.678573 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ad50fb5-d22e-4ef2-9e67-bb75a48f958c" containerName="sequence-graph-9f825" Sep 05 02:42:13.682718 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.682703 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:13.684908 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.684883 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-80c39-serving-cert\"" Sep 05 02:42:13.685031 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.684950 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 05 02:42:13.685031 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.684990 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7f65m\"" Sep 05 02:42:13.685213 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.685191 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-80c39-kube-rbac-proxy-sar-config\"" Sep 05 02:42:13.692101 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.692080 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt"] Sep 05 02:42:13.735457 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.735418 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e23711-1fcb-4efa-91ed-8ccf98539391-proxy-tls\") pod \"sequence-graph-80c39-76464f897d-75xtt\" (UID: \"49e23711-1fcb-4efa-91ed-8ccf98539391\") " pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:13.735457 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.735459 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e23711-1fcb-4efa-91ed-8ccf98539391-openshift-service-ca-bundle\") pod \"sequence-graph-80c39-76464f897d-75xtt\" (UID: \"49e23711-1fcb-4efa-91ed-8ccf98539391\") " pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:13.836007 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.835973 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e23711-1fcb-4efa-91ed-8ccf98539391-proxy-tls\") pod \"sequence-graph-80c39-76464f897d-75xtt\" (UID: \"49e23711-1fcb-4efa-91ed-8ccf98539391\") " pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:13.836007 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.836014 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e23711-1fcb-4efa-91ed-8ccf98539391-openshift-service-ca-bundle\") pod \"sequence-graph-80c39-76464f897d-75xtt\" (UID: \"49e23711-1fcb-4efa-91ed-8ccf98539391\") " pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:13.836251 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:42:13.836131 2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-80c39-serving-cert: secret "sequence-graph-80c39-serving-cert" not found Sep 05 02:42:13.836251 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:42:13.836197 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49e23711-1fcb-4efa-91ed-8ccf98539391-proxy-tls podName:49e23711-1fcb-4efa-91ed-8ccf98539391 nodeName:}" failed. No retries permitted until 2026-09-05 02:42:14.336179693 +0000 UTC m=+1457.822038652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/49e23711-1fcb-4efa-91ed-8ccf98539391-proxy-tls") pod "sequence-graph-80c39-76464f897d-75xtt" (UID: "49e23711-1fcb-4efa-91ed-8ccf98539391") : secret "sequence-graph-80c39-serving-cert" not found Sep 05 02:42:13.836657 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:13.836640 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e23711-1fcb-4efa-91ed-8ccf98539391-openshift-service-ca-bundle\") pod \"sequence-graph-80c39-76464f897d-75xtt\" (UID: \"49e23711-1fcb-4efa-91ed-8ccf98539391\") " pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:14.339777 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:14.339738 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e23711-1fcb-4efa-91ed-8ccf98539391-proxy-tls\") pod \"sequence-graph-80c39-76464f897d-75xtt\" (UID: \"49e23711-1fcb-4efa-91ed-8ccf98539391\") " pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:14.342248 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:14.342216 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e23711-1fcb-4efa-91ed-8ccf98539391-proxy-tls\") pod \"sequence-graph-80c39-76464f897d-75xtt\" (UID: \"49e23711-1fcb-4efa-91ed-8ccf98539391\") " pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:14.593714 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:14.593633 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:14.862960 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:14.862851 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt"] Sep 05 02:42:14.865704 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:42:14.865665 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e23711_1fcb_4efa_91ed_8ccf98539391.slice/crio-50e67da628c6474350398d085c98074806e26fa9eaf8af8ffd9ce37a486b9303 WatchSource:0}: Error finding container 50e67da628c6474350398d085c98074806e26fa9eaf8af8ffd9ce37a486b9303: Status 404 returned error can't find the container with id 50e67da628c6474350398d085c98074806e26fa9eaf8af8ffd9ce37a486b9303 Sep 05 02:42:15.772666 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:15.772625 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" event={"ID":"49e23711-1fcb-4efa-91ed-8ccf98539391","Type":"ContainerStarted","Data":"7addbacba36af374057e3a34df24ba4d0e9eacb68bccc8aac431d0050faef0b1"} Sep 05 02:42:15.772864 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:15.772701 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" event={"ID":"49e23711-1fcb-4efa-91ed-8ccf98539391","Type":"ContainerStarted","Data":"50e67da628c6474350398d085c98074806e26fa9eaf8af8ffd9ce37a486b9303"} Sep 05 02:42:15.772864 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:15.772728 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:15.791464 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:15.791399 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" podStartSLOduration=2.791379247 podStartE2EDuration="2.791379247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:42:15.790196454 +0000 UTC m=+1459.276055435" watchObservedRunningTime="2026-09-05 02:42:15.791379247 +0000 UTC m=+1459.277238223" Sep 05 02:42:21.783219 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:21.783192 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:23.780815 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:23.780775 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt"] Sep 05 02:42:23.781274 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:23.781007 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" containerID="cri-o://7addbacba36af374057e3a34df24ba4d0e9eacb68bccc8aac431d0050faef0b1" gracePeriod=30 Sep 05 02:42:26.781516 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:26.781450 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:42:31.780912 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:31.780873 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:42:34.101772 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.101743 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6"] Sep 05 02:42:34.104846 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.104829 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:34.107218 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.107196 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-9755b-serving-cert\"" Sep 05 02:42:34.107348 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.107198 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-9755b-kube-rbac-proxy-sar-config\"" Sep 05 02:42:34.113439 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.113417 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6"] Sep 05 02:42:34.197763 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.197710 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-proxy-tls\") pod \"ensemble-graph-9755b-f5958d5c6-nrxg6\" (UID: \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\") " pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:34.197964 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.197830 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-openshift-service-ca-bundle\") pod \"ensemble-graph-9755b-f5958d5c6-nrxg6\" (UID: \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\") " pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:34.298336 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.298305 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-proxy-tls\") pod \"ensemble-graph-9755b-f5958d5c6-nrxg6\" (UID: \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\") " pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:34.298523 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.298385 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-openshift-service-ca-bundle\") pod \"ensemble-graph-9755b-f5958d5c6-nrxg6\" (UID: \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\") " pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:34.299004 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.298982 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-openshift-service-ca-bundle\") pod \"ensemble-graph-9755b-f5958d5c6-nrxg6\" (UID: \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\") " pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:34.300777 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.300760 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-proxy-tls\") pod \"ensemble-graph-9755b-f5958d5c6-nrxg6\" (UID: \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\") " pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:34.416342 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.416231 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:34.687806 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.687723 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6"] Sep 05 02:42:34.691749 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:42:34.691723 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b51e7a9_9e28_45d1_89da_e4d4a8a65926.slice/crio-f45f3d03bbf988c9f82c0bc31c56c2b68fb255bf35c12d6cb9553437caa7c199 WatchSource:0}: Error finding container f45f3d03bbf988c9f82c0bc31c56c2b68fb255bf35c12d6cb9553437caa7c199: Status 404 returned error can't find the container with id f45f3d03bbf988c9f82c0bc31c56c2b68fb255bf35c12d6cb9553437caa7c199 Sep 05 02:42:34.846997 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.846962 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" event={"ID":"7b51e7a9-9e28-45d1-89da-e4d4a8a65926","Type":"ContainerStarted","Data":"ae8514239eca5cfe86f89adc6568b9b292188773059352c89e4bbc9728ed60c4"} Sep 05 02:42:34.847179 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.847014 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" event={"ID":"7b51e7a9-9e28-45d1-89da-e4d4a8a65926","Type":"ContainerStarted","Data":"f45f3d03bbf988c9f82c0bc31c56c2b68fb255bf35c12d6cb9553437caa7c199"} Sep 05 02:42:34.847179 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.847094 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:34.863893 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:34.863846 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" podStartSLOduration=0.863831205 podStartE2EDuration="863.831205ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:42:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:42:34.862451208 +0000 UTC m=+1478.348310183" watchObservedRunningTime="2026-09-05 02:42:34.863831205 +0000 UTC m=+1478.349690248" Sep 05 02:42:36.781512 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:36.781466 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:42:36.781897 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:36.781590 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:40.856555 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:40.856521 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:42:41.781162 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:41.781127 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:42:46.780899 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:46.780855 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:42:51.780935 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:51.780889 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:42:53.920772 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:53.920743 2567 generic.go:356] "Generic (PLEG): container finished" podID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerID="7addbacba36af374057e3a34df24ba4d0e9eacb68bccc8aac431d0050faef0b1" exitCode=0 Sep 05 02:42:53.921124 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:53.920785 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" event={"ID":"49e23711-1fcb-4efa-91ed-8ccf98539391","Type":"ContainerDied","Data":"7addbacba36af374057e3a34df24ba4d0e9eacb68bccc8aac431d0050faef0b1"} Sep 05 02:42:53.921124 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:53.920856 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" event={"ID":"49e23711-1fcb-4efa-91ed-8ccf98539391","Type":"ContainerDied","Data":"50e67da628c6474350398d085c98074806e26fa9eaf8af8ffd9ce37a486b9303"} Sep 05 02:42:53.921124 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:53.920886 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50e67da628c6474350398d085c98074806e26fa9eaf8af8ffd9ce37a486b9303" Sep 05 02:42:53.927436 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:53.927418 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:54.062702 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:54.062604 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e23711-1fcb-4efa-91ed-8ccf98539391-openshift-service-ca-bundle\") pod \"49e23711-1fcb-4efa-91ed-8ccf98539391\" (UID: \"49e23711-1fcb-4efa-91ed-8ccf98539391\") " Sep 05 02:42:54.062702 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:54.062648 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e23711-1fcb-4efa-91ed-8ccf98539391-proxy-tls\") pod \"49e23711-1fcb-4efa-91ed-8ccf98539391\" (UID: \"49e23711-1fcb-4efa-91ed-8ccf98539391\") " Sep 05 02:42:54.062997 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:54.062974 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e23711-1fcb-4efa-91ed-8ccf98539391-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "49e23711-1fcb-4efa-91ed-8ccf98539391" (UID: "49e23711-1fcb-4efa-91ed-8ccf98539391"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:42:54.065039 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:54.065011 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e23711-1fcb-4efa-91ed-8ccf98539391-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "49e23711-1fcb-4efa-91ed-8ccf98539391" (UID: "49e23711-1fcb-4efa-91ed-8ccf98539391"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:42:54.163606 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:54.163572 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e23711-1fcb-4efa-91ed-8ccf98539391-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:42:54.163606 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:54.163602 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e23711-1fcb-4efa-91ed-8ccf98539391-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:42:54.924408 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:54.924377 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt" Sep 05 02:42:54.943180 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:54.943147 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt"] Sep 05 02:42:54.947158 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:54.947131 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-80c39-76464f897d-75xtt"] Sep 05 02:42:55.017576 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:55.017545 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" path="/var/lib/kubelet/pods/49e23711-1fcb-4efa-91ed-8ccf98539391/volumes" Sep 05 02:42:57.020486 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:57.020458 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:42:57.023483 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:42:57.023463 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:43:34.013515 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.013465 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2"] Sep 05 02:43:34.013933 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.013788 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" Sep 05 02:43:34.013933 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.013800 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" Sep 05 02:43:34.013933 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.013861 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e23711-1fcb-4efa-91ed-8ccf98539391" containerName="sequence-graph-80c39" Sep 05 02:43:34.016740 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.016725 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:43:34.018948 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.018926 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-43d66-serving-cert\"" Sep 05 02:43:34.019065 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.018958 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-43d66-kube-rbac-proxy-sar-config\"" Sep 05 02:43:34.026126 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.026102 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2"] Sep 05 02:43:34.089931 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.089876 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86492a43-43e7-4f47-85e6-addac646479c-openshift-service-ca-bundle\") pod \"sequence-graph-43d66-7687b75cd7-xvcz2\" (UID: \"86492a43-43e7-4f47-85e6-addac646479c\") " pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:43:34.089931 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.089930 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86492a43-43e7-4f47-85e6-addac646479c-proxy-tls\") pod \"sequence-graph-43d66-7687b75cd7-xvcz2\" (UID: \"86492a43-43e7-4f47-85e6-addac646479c\") " pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:43:34.190686 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.190645 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86492a43-43e7-4f47-85e6-addac646479c-openshift-service-ca-bundle\") pod \"sequence-graph-43d66-7687b75cd7-xvcz2\" (UID: \"86492a43-43e7-4f47-85e6-addac646479c\") " pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:43:34.190686 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.190684 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86492a43-43e7-4f47-85e6-addac646479c-proxy-tls\") pod \"sequence-graph-43d66-7687b75cd7-xvcz2\" (UID: \"86492a43-43e7-4f47-85e6-addac646479c\") " pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:43:34.191312 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.191284 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86492a43-43e7-4f47-85e6-addac646479c-openshift-service-ca-bundle\") pod \"sequence-graph-43d66-7687b75cd7-xvcz2\" (UID: \"86492a43-43e7-4f47-85e6-addac646479c\") " pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:43:34.193204 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.193182 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86492a43-43e7-4f47-85e6-addac646479c-proxy-tls\") pod \"sequence-graph-43d66-7687b75cd7-xvcz2\" (UID: \"86492a43-43e7-4f47-85e6-addac646479c\") " pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:43:34.326928 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.326833 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:43:34.599908 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:34.599837 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2"] Sep 05 02:43:34.603078 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:43:34.603044 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86492a43_43e7_4f47_85e6_addac646479c.slice/crio-86c7019206a456f4a481b73a1fa375cbcbd91ee224cabe5b8ae21954b2ffd4a9 WatchSource:0}: Error finding container 86c7019206a456f4a481b73a1fa375cbcbd91ee224cabe5b8ae21954b2ffd4a9: Status 404 returned error can't find the container with id 86c7019206a456f4a481b73a1fa375cbcbd91ee224cabe5b8ae21954b2ffd4a9 Sep 05 02:43:35.078402 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:35.078372 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" event={"ID":"86492a43-43e7-4f47-85e6-addac646479c","Type":"ContainerStarted","Data":"fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025"} Sep 05 02:43:35.078402 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:35.078418 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" event={"ID":"86492a43-43e7-4f47-85e6-addac646479c","Type":"ContainerStarted","Data":"86c7019206a456f4a481b73a1fa375cbcbd91ee224cabe5b8ae21954b2ffd4a9"} Sep 05 02:43:35.078882 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:35.078540 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:43:35.096542 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:35.096486 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" podStartSLOduration=2.096473802 podStartE2EDuration="2.096473802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:43:35.095689783 +0000 UTC m=+1538.581548759" watchObservedRunningTime="2026-09-05 02:43:35.096473802 +0000 UTC m=+1538.582332777" Sep 05 02:43:41.088176 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:43:41.088147 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:47:57.049455 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:47:57.049421 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:47:57.052940 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:47:57.052919 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:48:57.101556 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:48:57.101523 2567 scope.go:117] "RemoveContainer" containerID="7addbacba36af374057e3a34df24ba4d0e9eacb68bccc8aac431d0050faef0b1" Sep 05 02:50:48.830624 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:50:48.830541 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6"] Sep 05 02:50:48.831163 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:50:48.830789 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" containerID="cri-o://ae8514239eca5cfe86f89adc6568b9b292188773059352c89e4bbc9728ed60c4" gracePeriod=30 Sep 05 02:50:50.854969 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:50:50.854925 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:50:55.855303 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:50:55.855259 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:51:00.854712 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:00.854675 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:51:00.855120 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:00.854797 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:51:05.854992 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:05.854957 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:51:10.855083 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:10.855045 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:51:15.855209 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:15.855170 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:51:18.884680 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:18.884647 2567 generic.go:356] "Generic (PLEG): container finished" podID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerID="ae8514239eca5cfe86f89adc6568b9b292188773059352c89e4bbc9728ed60c4" exitCode=0 Sep 05 02:51:18.885022 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:18.884714 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" event={"ID":"7b51e7a9-9e28-45d1-89da-e4d4a8a65926","Type":"ContainerDied","Data":"ae8514239eca5cfe86f89adc6568b9b292188773059352c89e4bbc9728ed60c4"} Sep 05 02:51:18.972041 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:18.972018 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:51:19.065970 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.065936 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-openshift-service-ca-bundle\") pod \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\" (UID: \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\") " Sep 05 02:51:19.065970 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.065977 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-proxy-tls\") pod \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\" (UID: \"7b51e7a9-9e28-45d1-89da-e4d4a8a65926\") " Sep 05 02:51:19.066348 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.066322 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "7b51e7a9-9e28-45d1-89da-e4d4a8a65926" (UID: "7b51e7a9-9e28-45d1-89da-e4d4a8a65926"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:51:19.068172 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.068148 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7b51e7a9-9e28-45d1-89da-e4d4a8a65926" (UID: "7b51e7a9-9e28-45d1-89da-e4d4a8a65926"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:51:19.167574 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.167473 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:51:19.167574 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.167519 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7b51e7a9-9e28-45d1-89da-e4d4a8a65926-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:51:19.890121 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.890084 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" event={"ID":"7b51e7a9-9e28-45d1-89da-e4d4a8a65926","Type":"ContainerDied","Data":"f45f3d03bbf988c9f82c0bc31c56c2b68fb255bf35c12d6cb9553437caa7c199"} Sep 05 02:51:19.890648 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.890143 2567 scope.go:117] "RemoveContainer" containerID="ae8514239eca5cfe86f89adc6568b9b292188773059352c89e4bbc9728ed60c4" Sep 05 02:51:19.890648 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.890100 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6" Sep 05 02:51:19.911144 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.911115 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6"] Sep 05 02:51:19.914836 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:19.914815 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9755b-f5958d5c6-nrxg6"] Sep 05 02:51:21.019595 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:21.019563 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" path="/var/lib/kubelet/pods/7b51e7a9-9e28-45d1-89da-e4d4a8a65926/volumes" Sep 05 02:51:48.712954 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:48.712921 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2"] Sep 05 02:51:48.713338 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:48.713159 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" containerID="cri-o://fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025" gracePeriod=30 Sep 05 02:51:49.064287 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.064205 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss"] Sep 05 02:51:49.064524 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.064511 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" Sep 05 02:51:49.064594 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.064526 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" Sep 05 02:51:49.064667 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.064645 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b51e7a9-9e28-45d1-89da-e4d4a8a65926" containerName="ensemble-graph-9755b" Sep 05 02:51:49.067571 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.067553 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:49.070791 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.070761 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-ca9bf-serving-cert\"" Sep 05 02:51:49.070904 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.070767 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-ca9bf-kube-rbac-proxy-sar-config\"" Sep 05 02:51:49.076984 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.076958 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss"] Sep 05 02:51:49.122934 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.122900 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9ed740e-d9e9-48ac-b51f-030724afe69a-proxy-tls\") pod \"splitter-graph-ca9bf-58997fc8cf-zg8ss\" (UID: \"f9ed740e-d9e9-48ac-b51f-030724afe69a\") " pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:49.123101 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.122952 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9ed740e-d9e9-48ac-b51f-030724afe69a-openshift-service-ca-bundle\") pod \"splitter-graph-ca9bf-58997fc8cf-zg8ss\" (UID: \"f9ed740e-d9e9-48ac-b51f-030724afe69a\") " pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:49.223487 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.223453 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9ed740e-d9e9-48ac-b51f-030724afe69a-openshift-service-ca-bundle\") pod \"splitter-graph-ca9bf-58997fc8cf-zg8ss\" (UID: \"f9ed740e-d9e9-48ac-b51f-030724afe69a\") " pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:49.223666 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.223527 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9ed740e-d9e9-48ac-b51f-030724afe69a-proxy-tls\") pod \"splitter-graph-ca9bf-58997fc8cf-zg8ss\" (UID: \"f9ed740e-d9e9-48ac-b51f-030724afe69a\") " pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:49.223666 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:51:49.223639 2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-ca9bf-serving-cert: secret "splitter-graph-ca9bf-serving-cert" not found Sep 05 02:51:49.223741 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:51:49.223702 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ed740e-d9e9-48ac-b51f-030724afe69a-proxy-tls podName:f9ed740e-d9e9-48ac-b51f-030724afe69a nodeName:}" failed. No retries permitted until 2026-09-05 02:51:49.723685402 +0000 UTC m=+2033.209544355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f9ed740e-d9e9-48ac-b51f-030724afe69a-proxy-tls") pod "splitter-graph-ca9bf-58997fc8cf-zg8ss" (UID: "f9ed740e-d9e9-48ac-b51f-030724afe69a") : secret "splitter-graph-ca9bf-serving-cert" not found Sep 05 02:51:49.224070 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.224049 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9ed740e-d9e9-48ac-b51f-030724afe69a-openshift-service-ca-bundle\") pod \"splitter-graph-ca9bf-58997fc8cf-zg8ss\" (UID: \"f9ed740e-d9e9-48ac-b51f-030724afe69a\") " pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:49.726890 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.726852 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9ed740e-d9e9-48ac-b51f-030724afe69a-proxy-tls\") pod \"splitter-graph-ca9bf-58997fc8cf-zg8ss\" (UID: \"f9ed740e-d9e9-48ac-b51f-030724afe69a\") " pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:49.729487 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.729462 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9ed740e-d9e9-48ac-b51f-030724afe69a-proxy-tls\") pod \"splitter-graph-ca9bf-58997fc8cf-zg8ss\" (UID: \"f9ed740e-d9e9-48ac-b51f-030724afe69a\") " pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:49.979177 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:49.979091 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:50.249466 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:50.249335 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss"] Sep 05 02:51:50.252110 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:51:50.252084 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9ed740e_d9e9_48ac_b51f_030724afe69a.slice/crio-c640aa3674967915cd61a81e5ba923bb4f725346becade6700fd656fc1b28aaf WatchSource:0}: Error finding container c640aa3674967915cd61a81e5ba923bb4f725346becade6700fd656fc1b28aaf: Status 404 returned error can't find the container with id c640aa3674967915cd61a81e5ba923bb4f725346becade6700fd656fc1b28aaf Sep 05 02:51:51.007577 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:51.007542 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" event={"ID":"f9ed740e-d9e9-48ac-b51f-030724afe69a","Type":"ContainerStarted","Data":"d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a"} Sep 05 02:51:51.007997 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:51.007597 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" event={"ID":"f9ed740e-d9e9-48ac-b51f-030724afe69a","Type":"ContainerStarted","Data":"c640aa3674967915cd61a81e5ba923bb4f725346becade6700fd656fc1b28aaf"} Sep 05 02:51:51.007997 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:51.007722 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:51.024638 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:51.024592 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" podStartSLOduration=2.024573626 podStartE2EDuration="2.024573626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:51:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:51:51.02234603 +0000 UTC m=+2034.508205005" watchObservedRunningTime="2026-09-05 02:51:51.024573626 +0000 UTC m=+2034.510432598" Sep 05 02:51:51.086147 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:51.086116 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:51:56.086781 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:56.086744 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:51:57.018482 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:57.018453 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:51:59.154959 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:59.154870 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss"] Sep 05 02:51:59.155376 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:51:59.155058 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" containerID="cri-o://d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a" gracePeriod=30 Sep 05 02:52:01.086198 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:01.086158 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:01.086576 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:01.086253 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:52:02.016093 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:02.016056 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:06.085921 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:06.085876 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:07.016187 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:07.016153 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:11.085833 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:11.085797 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:12.015940 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:12.015899 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:12.016122 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:12.015999 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:52:16.086558 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:16.086516 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:17.016296 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:17.016253 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:18.854959 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:18.854930 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:52:18.975979 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:18.975887 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86492a43-43e7-4f47-85e6-addac646479c-proxy-tls\") pod \"86492a43-43e7-4f47-85e6-addac646479c\" (UID: \"86492a43-43e7-4f47-85e6-addac646479c\") " Sep 05 02:52:18.975979 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:18.975960 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86492a43-43e7-4f47-85e6-addac646479c-openshift-service-ca-bundle\") pod \"86492a43-43e7-4f47-85e6-addac646479c\" (UID: \"86492a43-43e7-4f47-85e6-addac646479c\") " Sep 05 02:52:18.976320 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:18.976296 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86492a43-43e7-4f47-85e6-addac646479c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "86492a43-43e7-4f47-85e6-addac646479c" (UID: "86492a43-43e7-4f47-85e6-addac646479c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:52:18.978176 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:18.978150 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86492a43-43e7-4f47-85e6-addac646479c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "86492a43-43e7-4f47-85e6-addac646479c" (UID: "86492a43-43e7-4f47-85e6-addac646479c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:52:19.076616 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.076582 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86492a43-43e7-4f47-85e6-addac646479c-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:52:19.076616 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.076612 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86492a43-43e7-4f47-85e6-addac646479c-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:52:19.113588 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.113559 2567 generic.go:356] "Generic (PLEG): container finished" podID="86492a43-43e7-4f47-85e6-addac646479c" containerID="fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025" exitCode=0 Sep 05 02:52:19.113748 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.113598 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" event={"ID":"86492a43-43e7-4f47-85e6-addac646479c","Type":"ContainerDied","Data":"fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025"} Sep 05 02:52:19.113748 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.113608 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" Sep 05 02:52:19.113748 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.113636 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2" event={"ID":"86492a43-43e7-4f47-85e6-addac646479c","Type":"ContainerDied","Data":"86c7019206a456f4a481b73a1fa375cbcbd91ee224cabe5b8ae21954b2ffd4a9"} Sep 05 02:52:19.113748 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.113658 2567 scope.go:117] "RemoveContainer" containerID="fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025" Sep 05 02:52:19.121860 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.121842 2567 scope.go:117] "RemoveContainer" containerID="fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025" Sep 05 02:52:19.122172 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:52:19.122150 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025\": container with ID starting with fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025 not found: ID does not exist" containerID="fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025" Sep 05 02:52:19.122241 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.122185 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025"} err="failed to get container status \"fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025\": rpc error: code = NotFound desc = could not find container \"fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025\": container with ID starting with fa18776d9d2cd6d38724ede4a9874024f2554f548c20aa3cfe02b2c8e35a7025 not found: ID does not exist" Sep 05 02:52:19.130191 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.130166 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2"] Sep 05 02:52:19.134071 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:19.134048 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-43d66-7687b75cd7-xvcz2"] Sep 05 02:52:21.016902 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:21.016866 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86492a43-43e7-4f47-85e6-addac646479c" path="/var/lib/kubelet/pods/86492a43-43e7-4f47-85e6-addac646479c/volumes" Sep 05 02:52:22.015853 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:22.015815 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:27.015693 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:27.015653 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 02:52:29.309002 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:29.308969 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:52:29.465075 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:29.465042 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9ed740e-d9e9-48ac-b51f-030724afe69a-proxy-tls\") pod \"f9ed740e-d9e9-48ac-b51f-030724afe69a\" (UID: \"f9ed740e-d9e9-48ac-b51f-030724afe69a\") " Sep 05 02:52:29.465269 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:29.465117 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9ed740e-d9e9-48ac-b51f-030724afe69a-openshift-service-ca-bundle\") pod \"f9ed740e-d9e9-48ac-b51f-030724afe69a\" (UID: \"f9ed740e-d9e9-48ac-b51f-030724afe69a\") " Sep 05 02:52:29.465454 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:29.465427 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9ed740e-d9e9-48ac-b51f-030724afe69a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "f9ed740e-d9e9-48ac-b51f-030724afe69a" (UID: "f9ed740e-d9e9-48ac-b51f-030724afe69a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 02:52:29.467322 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:29.467301 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9ed740e-d9e9-48ac-b51f-030724afe69a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f9ed740e-d9e9-48ac-b51f-030724afe69a" (UID: "f9ed740e-d9e9-48ac-b51f-030724afe69a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 02:52:29.566537 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:29.566477 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9ed740e-d9e9-48ac-b51f-030724afe69a-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:52:29.566537 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:29.566532 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f9ed740e-d9e9-48ac-b51f-030724afe69a-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 02:52:30.159571 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:30.159538 2567 generic.go:356] "Generic (PLEG): container finished" podID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerID="d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a" exitCode=0 Sep 05 02:52:30.159923 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:30.159584 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" Sep 05 02:52:30.159923 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:30.159623 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" event={"ID":"f9ed740e-d9e9-48ac-b51f-030724afe69a","Type":"ContainerDied","Data":"d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a"} Sep 05 02:52:30.159923 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:30.159684 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss" event={"ID":"f9ed740e-d9e9-48ac-b51f-030724afe69a","Type":"ContainerDied","Data":"c640aa3674967915cd61a81e5ba923bb4f725346becade6700fd656fc1b28aaf"} Sep 05 02:52:30.159923 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:30.159712 2567 scope.go:117] "RemoveContainer" containerID="d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a" Sep 05 02:52:30.168143 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:30.168125 2567 scope.go:117] "RemoveContainer" containerID="d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a" Sep 05 02:52:30.168396 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:52:30.168377 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a\": container with ID starting with d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a not found: ID does not exist" containerID="d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a" Sep 05 02:52:30.168455 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:30.168405 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a"} err="failed to get container status \"d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a\": rpc error: code = NotFound desc = could not find container \"d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a\": container with ID starting with d02b3eef453ce4888cf621e8b4b60ea1d97888f190ac602785c38358848b866a not found: ID does not exist" Sep 05 02:52:30.179401 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:30.179379 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss"] Sep 05 02:52:30.182785 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:30.182760 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ca9bf-58997fc8cf-zg8ss"] Sep 05 02:52:31.017644 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:31.017612 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" path="/var/lib/kubelet/pods/f9ed740e-d9e9-48ac-b51f-030724afe69a/volumes" Sep 05 02:52:48.949628 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.949591 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd"] Sep 05 02:52:48.950006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.949889 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" Sep 05 02:52:48.950006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.949911 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" Sep 05 02:52:48.950006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.949931 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" Sep 05 02:52:48.950006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.949936 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" Sep 05 02:52:48.950006 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.949989 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9ed740e-d9e9-48ac-b51f-030724afe69a" containerName="splitter-graph-ca9bf" Sep 05 02:52:48.950169 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.950027 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="86492a43-43e7-4f47-85e6-addac646479c" containerName="sequence-graph-43d66" Sep 05 02:52:48.952906 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.952890 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:48.955200 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.955167 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 05 02:52:48.955353 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.955305 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7f65m\"" Sep 05 02:52:48.955353 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.955340 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-3af6d-kube-rbac-proxy-sar-config\"" Sep 05 02:52:48.955463 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.955344 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-3af6d-serving-cert\"" Sep 05 02:52:48.960544 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:48.960521 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd"] Sep 05 02:52:49.019153 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:49.019119 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff5b940-e570-413c-8af1-e3c871a1406c-openshift-service-ca-bundle\") pod \"switch-graph-3af6d-55559fdf56-6zztd\" (UID: \"cff5b940-e570-413c-8af1-e3c871a1406c\") " pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:49.019153 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:49.019152 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cff5b940-e570-413c-8af1-e3c871a1406c-proxy-tls\") pod \"switch-graph-3af6d-55559fdf56-6zztd\" (UID: \"cff5b940-e570-413c-8af1-e3c871a1406c\") " pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:49.120441 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:49.120394 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff5b940-e570-413c-8af1-e3c871a1406c-openshift-service-ca-bundle\") pod \"switch-graph-3af6d-55559fdf56-6zztd\" (UID: \"cff5b940-e570-413c-8af1-e3c871a1406c\") " pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:49.120441 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:49.120441 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cff5b940-e570-413c-8af1-e3c871a1406c-proxy-tls\") pod \"switch-graph-3af6d-55559fdf56-6zztd\" (UID: \"cff5b940-e570-413c-8af1-e3c871a1406c\") " pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:49.121090 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:49.121069 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff5b940-e570-413c-8af1-e3c871a1406c-openshift-service-ca-bundle\") pod \"switch-graph-3af6d-55559fdf56-6zztd\" (UID: \"cff5b940-e570-413c-8af1-e3c871a1406c\") " pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:49.123142 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:49.123120 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cff5b940-e570-413c-8af1-e3c871a1406c-proxy-tls\") pod \"switch-graph-3af6d-55559fdf56-6zztd\" (UID: \"cff5b940-e570-413c-8af1-e3c871a1406c\") " pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:49.264396 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:49.264318 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:49.535647 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:49.535566 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd"] Sep 05 02:52:49.538858 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:52:49.538834 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff5b940_e570_413c_8af1_e3c871a1406c.slice/crio-1dece69c6be6122a6a2ca2d4caf6e3428dc536aad4cd6b413f5c4d7f031b9719 WatchSource:0}: Error finding container 1dece69c6be6122a6a2ca2d4caf6e3428dc536aad4cd6b413f5c4d7f031b9719: Status 404 returned error can't find the container with id 1dece69c6be6122a6a2ca2d4caf6e3428dc536aad4cd6b413f5c4d7f031b9719 Sep 05 02:52:50.241995 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:50.241957 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" event={"ID":"cff5b940-e570-413c-8af1-e3c871a1406c","Type":"ContainerStarted","Data":"2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c"} Sep 05 02:52:50.242388 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:50.242011 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" event={"ID":"cff5b940-e570-413c-8af1-e3c871a1406c","Type":"ContainerStarted","Data":"1dece69c6be6122a6a2ca2d4caf6e3428dc536aad4cd6b413f5c4d7f031b9719"} Sep 05 02:52:50.242388 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:50.242049 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:50.259223 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:50.259166 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" podStartSLOduration=2.259149207 podStartE2EDuration="2.259149207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:52:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:52:50.258286504 +0000 UTC m=+2093.744145481" watchObservedRunningTime="2026-09-05 02:52:50.259149207 +0000 UTC m=+2093.745008199" Sep 05 02:52:56.252601 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:56.252568 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 02:52:57.073595 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:57.073566 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:52:57.076922 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:57.076899 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:52:59.398924 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.398892 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr"] Sep 05 02:52:59.403422 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.403403 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:52:59.405862 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.405841 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-a4a3e-serving-cert\"" Sep 05 02:52:59.405977 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.405936 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-a4a3e-kube-rbac-proxy-sar-config\"" Sep 05 02:52:59.410948 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.410926 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr"] Sep 05 02:52:59.503700 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.503672 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8456520-8f04-4fcb-9881-ce1c219c7272-proxy-tls\") pod \"splitter-graph-a4a3e-76cd97966c-7r5tr\" (UID: \"d8456520-8f04-4fcb-9881-ce1c219c7272\") " pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:52:59.503824 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.503703 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8456520-8f04-4fcb-9881-ce1c219c7272-openshift-service-ca-bundle\") pod \"splitter-graph-a4a3e-76cd97966c-7r5tr\" (UID: \"d8456520-8f04-4fcb-9881-ce1c219c7272\") " pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:52:59.604595 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.604564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8456520-8f04-4fcb-9881-ce1c219c7272-proxy-tls\") pod \"splitter-graph-a4a3e-76cd97966c-7r5tr\" (UID: \"d8456520-8f04-4fcb-9881-ce1c219c7272\") " pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:52:59.604716 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.604605 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8456520-8f04-4fcb-9881-ce1c219c7272-openshift-service-ca-bundle\") pod \"splitter-graph-a4a3e-76cd97966c-7r5tr\" (UID: \"d8456520-8f04-4fcb-9881-ce1c219c7272\") " pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:52:59.604716 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:52:59.604702 2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-a4a3e-serving-cert: secret "splitter-graph-a4a3e-serving-cert" not found Sep 05 02:52:59.604828 ip-10-0-128-63 kubenswrapper[2567]: E0905 02:52:59.604769 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8456520-8f04-4fcb-9881-ce1c219c7272-proxy-tls podName:d8456520-8f04-4fcb-9881-ce1c219c7272 nodeName:}" failed. No retries permitted until 2026-09-05 02:53:00.104751941 +0000 UTC m=+2103.590610894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d8456520-8f04-4fcb-9881-ce1c219c7272-proxy-tls") pod "splitter-graph-a4a3e-76cd97966c-7r5tr" (UID: "d8456520-8f04-4fcb-9881-ce1c219c7272") : secret "splitter-graph-a4a3e-serving-cert" not found Sep 05 02:52:59.605267 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:52:59.605245 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8456520-8f04-4fcb-9881-ce1c219c7272-openshift-service-ca-bundle\") pod \"splitter-graph-a4a3e-76cd97966c-7r5tr\" (UID: \"d8456520-8f04-4fcb-9881-ce1c219c7272\") " pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:53:00.107721 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:53:00.107683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8456520-8f04-4fcb-9881-ce1c219c7272-proxy-tls\") pod \"splitter-graph-a4a3e-76cd97966c-7r5tr\" (UID: \"d8456520-8f04-4fcb-9881-ce1c219c7272\") " pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:53:00.110271 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:53:00.110251 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8456520-8f04-4fcb-9881-ce1c219c7272-proxy-tls\") pod \"splitter-graph-a4a3e-76cd97966c-7r5tr\" (UID: \"d8456520-8f04-4fcb-9881-ce1c219c7272\") " pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:53:00.313849 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:53:00.313815 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:53:00.585322 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:53:00.584422 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr"] Sep 05 02:53:00.589653 ip-10-0-128-63 kubenswrapper[2567]: W0905 02:53:00.589627 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8456520_8f04_4fcb_9881_ce1c219c7272.slice/crio-bf6eb9e77fba4c74aeb1bf12740d0a67b4815d18f88da87360c4150305213256 WatchSource:0}: Error finding container bf6eb9e77fba4c74aeb1bf12740d0a67b4815d18f88da87360c4150305213256: Status 404 returned error can't find the container with id bf6eb9e77fba4c74aeb1bf12740d0a67b4815d18f88da87360c4150305213256 Sep 05 02:53:01.286698 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:53:01.286663 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" event={"ID":"d8456520-8f04-4fcb-9881-ce1c219c7272","Type":"ContainerStarted","Data":"4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db"} Sep 05 02:53:01.286896 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:53:01.286712 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" event={"ID":"d8456520-8f04-4fcb-9881-ce1c219c7272","Type":"ContainerStarted","Data":"bf6eb9e77fba4c74aeb1bf12740d0a67b4815d18f88da87360c4150305213256"} Sep 05 02:53:01.286896 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:53:01.286763 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:53:01.301761 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:53:01.301715 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" podStartSLOduration=2.301700781 podStartE2EDuration="2.301700781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 02:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 02:53:01.301185046 +0000 UTC m=+2104.787044021" watchObservedRunningTime="2026-09-05 02:53:01.301700781 +0000 UTC m=+2104.787559799" Sep 05 02:53:07.296325 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:53:07.296297 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 02:57:57.097418 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:57:57.097391 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 02:57:57.103261 ip-10-0-128-63 kubenswrapper[2567]: I0905 02:57:57.103233 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 03:01:13.951190 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:13.951157 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr"] Sep 05 03:01:13.951741 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:13.951364 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" containerID="cri-o://4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db" gracePeriod=30 Sep 05 03:01:17.294453 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:17.294409 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:01:22.294148 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:22.294102 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:01:27.294357 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:27.294319 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:01:27.294761 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:27.294418 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 03:01:32.294430 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:32.294389 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:01:37.294746 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:37.294703 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:01:42.294196 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:42.294157 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:01:44.091430 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.091098 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 03:01:44.223010 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.222925 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8456520-8f04-4fcb-9881-ce1c219c7272-openshift-service-ca-bundle\") pod \"d8456520-8f04-4fcb-9881-ce1c219c7272\" (UID: \"d8456520-8f04-4fcb-9881-ce1c219c7272\") " Sep 05 03:01:44.223010 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.222994 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8456520-8f04-4fcb-9881-ce1c219c7272-proxy-tls\") pod \"d8456520-8f04-4fcb-9881-ce1c219c7272\" (UID: \"d8456520-8f04-4fcb-9881-ce1c219c7272\") " Sep 05 03:01:44.223327 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.223302 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8456520-8f04-4fcb-9881-ce1c219c7272-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d8456520-8f04-4fcb-9881-ce1c219c7272" (UID: "d8456520-8f04-4fcb-9881-ce1c219c7272"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 03:01:44.225050 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.225025 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8456520-8f04-4fcb-9881-ce1c219c7272-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d8456520-8f04-4fcb-9881-ce1c219c7272" (UID: "d8456520-8f04-4fcb-9881-ce1c219c7272"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 03:01:44.282915 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.282885 2567 generic.go:356] "Generic (PLEG): container finished" podID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerID="4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db" exitCode=0 Sep 05 03:01:44.283055 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.282927 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" Sep 05 03:01:44.283055 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.282971 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" event={"ID":"d8456520-8f04-4fcb-9881-ce1c219c7272","Type":"ContainerDied","Data":"4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db"} Sep 05 03:01:44.283055 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.283031 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr" event={"ID":"d8456520-8f04-4fcb-9881-ce1c219c7272","Type":"ContainerDied","Data":"bf6eb9e77fba4c74aeb1bf12740d0a67b4815d18f88da87360c4150305213256"} Sep 05 03:01:44.283242 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.283060 2567 scope.go:117] "RemoveContainer" containerID="4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db" Sep 05 03:01:44.291205 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.291190 2567 scope.go:117] "RemoveContainer" containerID="4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db" Sep 05 03:01:44.291461 ip-10-0-128-63 kubenswrapper[2567]: E0905 03:01:44.291437 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db\": container with ID starting with 4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db not found: ID does not exist" containerID="4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db" Sep 05 03:01:44.291596 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.291470 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db"} err="failed to get container status \"4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db\": rpc error: code = NotFound desc = could not find container \"4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db\": container with ID starting with 4107ecde98c0331371a8630355a8a093fc0041888d41cfedb04a3e34588cf7db not found: ID does not exist" Sep 05 03:01:44.303378 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.303357 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr"] Sep 05 03:01:44.309534 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.309510 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a4a3e-76cd97966c-7r5tr"] Sep 05 03:01:44.324140 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.324105 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8456520-8f04-4fcb-9881-ce1c219c7272-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 03:01:44.324140 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:44.324139 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8456520-8f04-4fcb-9881-ce1c219c7272-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 03:01:45.016956 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:01:45.016913 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" path="/var/lib/kubelet/pods/d8456520-8f04-4fcb-9881-ce1c219c7272/volumes" Sep 05 03:02:57.123782 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:02:57.123695 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 03:02:57.135952 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:02:57.135917 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 03:07:57.154714 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:07:57.154687 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 03:07:57.161364 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:07:57.161337 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 03:09:07.839643 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:07.839556 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd"] Sep 05 03:09:07.840206 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:07.839830 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" containerID="cri-o://2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c" gracePeriod=30 Sep 05 03:09:08.808417 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.808380 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gnpzx/must-gather-ghv62"] Sep 05 03:09:08.808733 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.808719 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" Sep 05 03:09:08.808782 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.808735 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" Sep 05 03:09:08.808816 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.808798 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8456520-8f04-4fcb-9881-ce1c219c7272" containerName="splitter-graph-a4a3e" Sep 05 03:09:08.811911 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.811893 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:08.814132 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.814102 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gnpzx\"/\"default-dockercfg-2pgtn\"" Sep 05 03:09:08.814772 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.814754 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gnpzx\"/\"openshift-service-ca.crt\"" Sep 05 03:09:08.814851 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.814799 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gnpzx\"/\"kube-root-ca.crt\"" Sep 05 03:09:08.820343 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.820320 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gnpzx/must-gather-ghv62"] Sep 05 03:09:08.870799 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.870769 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjfc2\" (UniqueName: \"kubernetes.io/projected/d265cfa6-0162-471e-8139-64a700d54a09-kube-api-access-mjfc2\") pod \"must-gather-ghv62\" (UID: \"d265cfa6-0162-471e-8139-64a700d54a09\") " pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:08.871100 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.870803 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d265cfa6-0162-471e-8139-64a700d54a09-must-gather-output\") pod \"must-gather-ghv62\" (UID: \"d265cfa6-0162-471e-8139-64a700d54a09\") " pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:08.971456 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.971425 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjfc2\" (UniqueName: \"kubernetes.io/projected/d265cfa6-0162-471e-8139-64a700d54a09-kube-api-access-mjfc2\") pod \"must-gather-ghv62\" (UID: \"d265cfa6-0162-471e-8139-64a700d54a09\") " pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:08.971621 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.971465 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d265cfa6-0162-471e-8139-64a700d54a09-must-gather-output\") pod \"must-gather-ghv62\" (UID: \"d265cfa6-0162-471e-8139-64a700d54a09\") " pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:08.971823 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.971804 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d265cfa6-0162-471e-8139-64a700d54a09-must-gather-output\") pod \"must-gather-ghv62\" (UID: \"d265cfa6-0162-471e-8139-64a700d54a09\") " pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:08.979633 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:08.979609 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjfc2\" (UniqueName: \"kubernetes.io/projected/d265cfa6-0162-471e-8139-64a700d54a09-kube-api-access-mjfc2\") pod \"must-gather-ghv62\" (UID: \"d265cfa6-0162-471e-8139-64a700d54a09\") " pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:09.121570 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:09.121468 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:09.390971 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:09.390898 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gnpzx/must-gather-ghv62"] Sep 05 03:09:09.394009 ip-10-0-128-63 kubenswrapper[2567]: W0905 03:09:09.393984 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd265cfa6_0162_471e_8139_64a700d54a09.slice/crio-3e80a4a5cdc15f9209d0ac9fbcb3df56f110cc03b981b2906fcf6641ba4da167 WatchSource:0}: Error finding container 3e80a4a5cdc15f9209d0ac9fbcb3df56f110cc03b981b2906fcf6641ba4da167: Status 404 returned error can't find the container with id 3e80a4a5cdc15f9209d0ac9fbcb3df56f110cc03b981b2906fcf6641ba4da167 Sep 05 03:09:09.395993 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:09.395978 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 05 03:09:09.963361 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:09.963296 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gnpzx/must-gather-ghv62" event={"ID":"d265cfa6-0162-471e-8139-64a700d54a09","Type":"ContainerStarted","Data":"3e80a4a5cdc15f9209d0ac9fbcb3df56f110cc03b981b2906fcf6641ba4da167"} Sep 05 03:09:11.252761 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:11.252472 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:09:15.993273 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:15.993229 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gnpzx/must-gather-ghv62" event={"ID":"d265cfa6-0162-471e-8139-64a700d54a09","Type":"ContainerStarted","Data":"4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec"} Sep 05 03:09:15.993273 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:15.993279 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gnpzx/must-gather-ghv62" event={"ID":"d265cfa6-0162-471e-8139-64a700d54a09","Type":"ContainerStarted","Data":"d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93"} Sep 05 03:09:16.011650 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:16.011601 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gnpzx/must-gather-ghv62" podStartSLOduration=1.9095348090000002 podStartE2EDuration="8.011584105s" totalImagesPullingTime="6.102049296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 03:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 03:09:16.010606255 +0000 UTC m=+3079.496465230" watchObservedRunningTime="2026-09-05 03:09:16.011584105 +0000 UTC m=+3079.497443079" Sep 05 03:09:16.251702 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:16.251593 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:09:21.250519 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:21.250450 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:09:21.250916 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:21.250583 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 03:09:24.490528 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:24.490486 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:25.342159 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:25.342126 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:26.182844 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:26.182816 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:26.252176 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:26.252126 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:09:26.987766 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:26.987731 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:27.843005 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:27.842969 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:28.633528 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:28.633476 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:29.417508 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:29.417460 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:30.244286 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:30.244259 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:31.187243 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:31.187217 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:31.253070 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:31.252996 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:09:32.039064 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:32.039034 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:32.865117 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:32.865087 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:33.703135 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:33.703093 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:34.512159 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:34.512122 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:35.340960 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:35.340917 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-3af6d-55559fdf56-6zztd_cff5b940-e570-413c-8af1-e3c871a1406c/switch-graph-3af6d/0.log" Sep 05 03:09:36.090089 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:36.090054 2567 generic.go:356] "Generic (PLEG): container finished" podID="d265cfa6-0162-471e-8139-64a700d54a09" containerID="d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93" exitCode=0 Sep 05 03:09:36.090485 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:36.090114 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gnpzx/must-gather-ghv62" event={"ID":"d265cfa6-0162-471e-8139-64a700d54a09","Type":"ContainerDied","Data":"d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93"} Sep 05 03:09:36.090485 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:36.090404 2567 scope.go:117] "RemoveContainer" containerID="d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93" Sep 05 03:09:36.250611 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:36.250528 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 03:09:36.983134 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:36.983110 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gnpzx_must-gather-ghv62_d265cfa6-0162-471e-8139-64a700d54a09/gather/0.log" Sep 05 03:09:37.984134 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:37.984114 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 03:09:38.098027 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.097945 2567 generic.go:356] "Generic (PLEG): container finished" podID="cff5b940-e570-413c-8af1-e3c871a1406c" containerID="2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c" exitCode=0 Sep 05 03:09:38.098027 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.097993 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" Sep 05 03:09:38.098027 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.098002 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" event={"ID":"cff5b940-e570-413c-8af1-e3c871a1406c","Type":"ContainerDied","Data":"2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c"} Sep 05 03:09:38.098229 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.098043 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd" event={"ID":"cff5b940-e570-413c-8af1-e3c871a1406c","Type":"ContainerDied","Data":"1dece69c6be6122a6a2ca2d4caf6e3428dc536aad4cd6b413f5c4d7f031b9719"} Sep 05 03:09:38.098229 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.098064 2567 scope.go:117] "RemoveContainer" containerID="2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c" Sep 05 03:09:38.106096 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.106069 2567 scope.go:117] "RemoveContainer" containerID="2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c" Sep 05 03:09:38.106362 ip-10-0-128-63 kubenswrapper[2567]: E0905 03:09:38.106344 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c\": container with ID starting with 2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c not found: ID does not exist" containerID="2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c" Sep 05 03:09:38.106401 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.106372 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c"} err="failed to get container status \"2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c\": rpc error: code = NotFound desc = could not find container \"2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c\": container with ID starting with 2afec18e35c1f03a5095f8746e46c8476c8b73fe795c92d0ff8d8ec6de59c95c not found: ID does not exist" Sep 05 03:09:38.129037 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.129019 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff5b940-e570-413c-8af1-e3c871a1406c-openshift-service-ca-bundle\") pod \"cff5b940-e570-413c-8af1-e3c871a1406c\" (UID: \"cff5b940-e570-413c-8af1-e3c871a1406c\") " Sep 05 03:09:38.129122 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.129064 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cff5b940-e570-413c-8af1-e3c871a1406c-proxy-tls\") pod \"cff5b940-e570-413c-8af1-e3c871a1406c\" (UID: \"cff5b940-e570-413c-8af1-e3c871a1406c\") " Sep 05 03:09:38.129345 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.129326 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cff5b940-e570-413c-8af1-e3c871a1406c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "cff5b940-e570-413c-8af1-e3c871a1406c" (UID: "cff5b940-e570-413c-8af1-e3c871a1406c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 03:09:38.131195 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.131173 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cff5b940-e570-413c-8af1-e3c871a1406c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cff5b940-e570-413c-8af1-e3c871a1406c" (UID: "cff5b940-e570-413c-8af1-e3c871a1406c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 03:09:38.230616 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.230587 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff5b940-e570-413c-8af1-e3c871a1406c-openshift-service-ca-bundle\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 03:09:38.230616 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.230612 2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cff5b940-e570-413c-8af1-e3c871a1406c-proxy-tls\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 03:09:38.418800 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.418775 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd"] Sep 05 03:09:38.422484 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:38.422464 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-3af6d-55559fdf56-6zztd"] Sep 05 03:09:39.016729 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:39.016700 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" path="/var/lib/kubelet/pods/cff5b940-e570-413c-8af1-e3c871a1406c/volumes" Sep 05 03:09:40.231707 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:40.231676 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-74l8j_d398890b-b9a3-4b54-abdd-d0f90f315b66/global-pull-secret-syncer/0.log" Sep 05 03:09:40.398078 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:40.398050 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zgb7b_fd04d4e5-d457-482a-b1ee-a2d606c18b81/konnectivity-agent/0.log" Sep 05 03:09:40.425510 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:40.425469 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-63.ec2.internal_07df7873c55aad720a5c1b311be3ff60/haproxy/0.log" Sep 05 03:09:42.401070 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.401037 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-gnpzx/must-gather-ghv62"] Sep 05 03:09:42.401625 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.401207 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-gnpzx/must-gather-ghv62" podUID="d265cfa6-0162-471e-8139-64a700d54a09" containerName="copy" containerID="cri-o://4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec" gracePeriod=2 Sep 05 03:09:42.406173 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.406145 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-gnpzx/must-gather-ghv62"] Sep 05 03:09:42.628202 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.628180 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gnpzx_must-gather-ghv62_d265cfa6-0162-471e-8139-64a700d54a09/copy/0.log" Sep 05 03:09:42.630860 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.630842 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:42.632668 ip-10-0-128-63 kubenswrapper[2567]: E0905 03:09:42.632645 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-ghv62\" is forbidden: User \"system:node:ip-10-0-128-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gnpzx\": no relationship found between node 'ip-10-0-128-63.ec2.internal' and this object" podUID="d265cfa6-0162-471e-8139-64a700d54a09" pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:42.663022 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.662979 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjfc2\" (UniqueName: \"kubernetes.io/projected/d265cfa6-0162-471e-8139-64a700d54a09-kube-api-access-mjfc2\") pod \"d265cfa6-0162-471e-8139-64a700d54a09\" (UID: \"d265cfa6-0162-471e-8139-64a700d54a09\") " Sep 05 03:09:42.663022 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.663014 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d265cfa6-0162-471e-8139-64a700d54a09-must-gather-output\") pod \"d265cfa6-0162-471e-8139-64a700d54a09\" (UID: \"d265cfa6-0162-471e-8139-64a700d54a09\") " Sep 05 03:09:42.664574 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.664550 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d265cfa6-0162-471e-8139-64a700d54a09-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "d265cfa6-0162-471e-8139-64a700d54a09" (UID: "d265cfa6-0162-471e-8139-64a700d54a09"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 03:09:42.665137 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.665114 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d265cfa6-0162-471e-8139-64a700d54a09-kube-api-access-mjfc2" (OuterVolumeSpecName: "kube-api-access-mjfc2") pod "d265cfa6-0162-471e-8139-64a700d54a09" (UID: "d265cfa6-0162-471e-8139-64a700d54a09"). InnerVolumeSpecName "kube-api-access-mjfc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 03:09:42.764456 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.764422 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjfc2\" (UniqueName: \"kubernetes.io/projected/d265cfa6-0162-471e-8139-64a700d54a09-kube-api-access-mjfc2\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 03:09:42.764456 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:42.764452 2567 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d265cfa6-0162-471e-8139-64a700d54a09-must-gather-output\") on node \"ip-10-0-128-63.ec2.internal\" DevicePath \"\"" Sep 05 03:09:43.019828 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.019792 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d265cfa6-0162-471e-8139-64a700d54a09" path="/var/lib/kubelet/pods/d265cfa6-0162-471e-8139-64a700d54a09/volumes" Sep 05 03:09:43.117085 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.117059 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gnpzx_must-gather-ghv62_d265cfa6-0162-471e-8139-64a700d54a09/copy/0.log" Sep 05 03:09:43.117422 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.117401 2567 generic.go:356] "Generic (PLEG): container finished" podID="d265cfa6-0162-471e-8139-64a700d54a09" containerID="4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec" exitCode=143 Sep 05 03:09:43.117475 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.117443 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gnpzx/must-gather-ghv62" Sep 05 03:09:43.117475 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.117462 2567 scope.go:117] "RemoveContainer" containerID="4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec" Sep 05 03:09:43.124936 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.124916 2567 scope.go:117] "RemoveContainer" containerID="d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93" Sep 05 03:09:43.137086 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.137068 2567 scope.go:117] "RemoveContainer" containerID="4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec" Sep 05 03:09:43.137366 ip-10-0-128-63 kubenswrapper[2567]: E0905 03:09:43.137348 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec\": container with ID starting with 4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec not found: ID does not exist" containerID="4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec" Sep 05 03:09:43.137416 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.137374 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec"} err="failed to get container status \"4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec\": rpc error: code = NotFound desc = could not find container \"4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec\": container with ID starting with 4fbad0e8df0f42d13e59d8cbed2c1bf570c0a52ca72ad64f93afd65572c35dec not found: ID does not exist" Sep 05 03:09:43.137416 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.137390 2567 scope.go:117] "RemoveContainer" containerID="d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93" Sep 05 03:09:43.137677 ip-10-0-128-63 kubenswrapper[2567]: E0905 03:09:43.137658 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93\": container with ID starting with d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93 not found: ID does not exist" containerID="d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93" Sep 05 03:09:43.137740 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:43.137685 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93"} err="failed to get container status \"d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93\": rpc error: code = NotFound desc = could not find container \"d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93\": container with ID starting with d401a5eafffcb6d0eb3464a48fc7560b4109a60d3b99188b78afcf67819a3b93 not found: ID does not exist" Sep 05 03:09:44.696421 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:44.696342 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-lfv7f_9facd67b-62a7-4e19-b399-19b68d0d97a4/cluster-monitoring-operator/0.log" Sep 05 03:09:44.909340 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:44.909312 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4wqbc_0361429b-11bd-4c00-814e-ee5bb97d77a8/node-exporter/0.log" Sep 05 03:09:44.939481 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:44.939464 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4wqbc_0361429b-11bd-4c00-814e-ee5bb97d77a8/kube-rbac-proxy/0.log" Sep 05 03:09:44.972089 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:44.972072 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4wqbc_0361429b-11bd-4c00-814e-ee5bb97d77a8/init-textfile/0.log" Sep 05 03:09:45.307227 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.307161 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aceeb27b-700f-4df1-b20a-4b0e349a1878/prometheus/0.log" Sep 05 03:09:45.329994 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.329971 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aceeb27b-700f-4df1-b20a-4b0e349a1878/config-reloader/0.log" Sep 05 03:09:45.353362 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.353338 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aceeb27b-700f-4df1-b20a-4b0e349a1878/thanos-sidecar/0.log" Sep 05 03:09:45.384819 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.384794 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aceeb27b-700f-4df1-b20a-4b0e349a1878/kube-rbac-proxy-web/0.log" Sep 05 03:09:45.415135 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.415113 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aceeb27b-700f-4df1-b20a-4b0e349a1878/kube-rbac-proxy/0.log" Sep 05 03:09:45.448799 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.448777 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aceeb27b-700f-4df1-b20a-4b0e349a1878/kube-rbac-proxy-thanos/0.log" Sep 05 03:09:45.479062 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.479033 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aceeb27b-700f-4df1-b20a-4b0e349a1878/init-config-reloader/0.log" Sep 05 03:09:45.525010 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.524985 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-hgh8g_09810df9-1b98-489b-89a8-a00f04328c4a/prometheus-operator/0.log" Sep 05 03:09:45.549543 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.549522 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-hgh8g_09810df9-1b98-489b-89a8-a00f04328c4a/kube-rbac-proxy/0.log" Sep 05 03:09:45.582130 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:45.582068 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-dbbjx_c9f798c1-bd10-4bce-b33d-b12ac31ee286/prometheus-operator-admission-webhook/0.log" Sep 05 03:09:47.639291 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:47.639261 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/2.log" Sep 05 03:09:47.643623 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:47.643590 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cfg6p_af6ef192-9048-4fb7-9b65-b2de6f8a7048/console-operator/3.log" Sep 05 03:09:48.239429 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239398 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj"] Sep 05 03:09:48.239713 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239701 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d265cfa6-0162-471e-8139-64a700d54a09" containerName="copy" Sep 05 03:09:48.239769 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239715 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d265cfa6-0162-471e-8139-64a700d54a09" containerName="copy" Sep 05 03:09:48.239769 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239730 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" Sep 05 03:09:48.239769 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239735 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" Sep 05 03:09:48.239769 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239744 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d265cfa6-0162-471e-8139-64a700d54a09" containerName="gather" Sep 05 03:09:48.239769 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239749 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d265cfa6-0162-471e-8139-64a700d54a09" containerName="gather" Sep 05 03:09:48.239938 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239793 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cff5b940-e570-413c-8af1-e3c871a1406c" containerName="switch-graph-3af6d" Sep 05 03:09:48.239938 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239800 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d265cfa6-0162-471e-8139-64a700d54a09" containerName="copy" Sep 05 03:09:48.239938 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.239806 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d265cfa6-0162-471e-8139-64a700d54a09" containerName="gather" Sep 05 03:09:48.244742 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.244722 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.247113 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.247090 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zkkfn\"/\"kube-root-ca.crt\"" Sep 05 03:09:48.247226 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.247197 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zkkfn\"/\"openshift-service-ca.crt\"" Sep 05 03:09:48.247999 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.247984 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zkkfn\"/\"default-dockercfg-5rzbt\"" Sep 05 03:09:48.252678 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.252654 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj"] Sep 05 03:09:48.307684 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.307655 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-podres\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.307827 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.307695 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-sys\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.307827 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.307717 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-lib-modules\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.307827 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.307752 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh62d\" (UniqueName: \"kubernetes.io/projected/3e9c637d-840e-484e-94ed-3ad126fa7c73-kube-api-access-gh62d\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.307827 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.307776 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-proc\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.408748 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.408719 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-podres\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.408895 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.408761 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-sys\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.408895 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.408784 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-lib-modules\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.408895 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.408814 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh62d\" (UniqueName: \"kubernetes.io/projected/3e9c637d-840e-484e-94ed-3ad126fa7c73-kube-api-access-gh62d\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.408895 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.408841 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-proc\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.408895 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.408846 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-sys\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.408895 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.408875 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-podres\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.408895 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.408893 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-proc\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.409155 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.408920 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e9c637d-840e-484e-94ed-3ad126fa7c73-lib-modules\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.417082 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.417055 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh62d\" (UniqueName: \"kubernetes.io/projected/3e9c637d-840e-484e-94ed-3ad126fa7c73-kube-api-access-gh62d\") pod \"perf-node-gather-daemonset-6rlzj\" (UID: \"3e9c637d-840e-484e-94ed-3ad126fa7c73\") " pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.554559 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.554450 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:48.632918 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.632894 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-sj2lm_8e504a92-8f83-4d1b-91b6-c3ef4babbaaf/volume-data-source-validator/0.log" Sep 05 03:09:48.823683 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:48.823593 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj"] Sep 05 03:09:48.826412 ip-10-0-128-63 kubenswrapper[2567]: W0905 03:09:48.826386 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3e9c637d_840e_484e_94ed_3ad126fa7c73.slice/crio-f32e520198659a91e3cebebc6e9e2f94a32f1dd9ee4216751d3f11c7f17f79e0 WatchSource:0}: Error finding container f32e520198659a91e3cebebc6e9e2f94a32f1dd9ee4216751d3f11c7f17f79e0: Status 404 returned error can't find the container with id f32e520198659a91e3cebebc6e9e2f94a32f1dd9ee4216751d3f11c7f17f79e0 Sep 05 03:09:49.141838 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:49.141753 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" event={"ID":"3e9c637d-840e-484e-94ed-3ad126fa7c73","Type":"ContainerStarted","Data":"295e8bbc1bc25a1e1cc594dac6c9e9cf0dce07a11108768cca41b91f51f54210"} Sep 05 03:09:49.141838 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:49.141814 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" event={"ID":"3e9c637d-840e-484e-94ed-3ad126fa7c73","Type":"ContainerStarted","Data":"f32e520198659a91e3cebebc6e9e2f94a32f1dd9ee4216751d3f11c7f17f79e0"} Sep 05 03:09:49.142041 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:49.141856 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:49.159719 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:49.159668 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" podStartSLOduration=1.159655295 podStartE2EDuration="1.159655295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 03:09:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 03:09:49.158320437 +0000 UTC m=+3112.644179412" watchObservedRunningTime="2026-09-05 03:09:49.159655295 +0000 UTC m=+3112.645514271" Sep 05 03:09:49.599145 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:49.599116 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ghw92_b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc/dns/0.log" Sep 05 03:09:49.621094 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:49.621070 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ghw92_b4f5d9a6-4ee3-41c4-9be0-c57ed41451cc/kube-rbac-proxy/0.log" Sep 05 03:09:49.739585 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:49.739552 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p4p4n_9e6dbc91-2b3f-40a1-9b30-525617d8c100/dns-node-resolver/0.log" Sep 05 03:09:50.202154 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:50.202124 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-66cfff688c-45r6r_120f4523-946f-45dd-ba90-8ade102169f3/registry/0.log" Sep 05 03:09:50.226542 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:50.226515 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qx5mt_687f9036-0a6d-4fb4-abef-d64801928cc1/node-ca/0.log" Sep 05 03:09:51.450394 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:51.450363 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wnvsv_4a089884-83b9-4183-a0d5-3d1f2dc10ba3/serve-healthcheck-canary/0.log" Sep 05 03:09:51.861437 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:51.861405 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-92bgx_0b525226-203f-4f6f-91ed-93f07eb5653f/insights-operator/0.log" Sep 05 03:09:51.862502 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:51.862473 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-92bgx_0b525226-203f-4f6f-91ed-93f07eb5653f/insights-operator/1.log" Sep 05 03:09:51.977998 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:51.977966 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hp58v_611d2010-264b-444d-8bf2-f856af766d89/kube-rbac-proxy/0.log" Sep 05 03:09:52.003798 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:52.003770 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hp58v_611d2010-264b-444d-8bf2-f856af766d89/exporter/0.log" Sep 05 03:09:52.028867 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:52.028846 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hp58v_611d2010-264b-444d-8bf2-f856af766d89/extractor/0.log" Sep 05 03:09:55.155046 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:55.155015 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zkkfn/perf-node-gather-daemonset-6rlzj" Sep 05 03:09:59.342362 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:59.342336 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-rr89z_0b4e3d1f-b577-414a-a0e4-882b83ae9d45/kube-storage-version-migrator-operator/1.log" Sep 05 03:09:59.343457 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:09:59.343431 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-rr89z_0b4e3d1f-b577-414a-a0e4-882b83ae9d45/kube-storage-version-migrator-operator/0.log" Sep 05 03:10:00.309541 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:00.309510 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5bshb_7a211ccd-3d23-4de1-aa38-b4a3d2b3c4bb/kube-multus/0.log" Sep 05 03:10:00.715748 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:00.715703 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5q89w_75cd35e8-1c2c-4bc6-a121-15adc834eb69/kube-multus-additional-cni-plugins/0.log" Sep 05 03:10:00.739380 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:00.739357 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5q89w_75cd35e8-1c2c-4bc6-a121-15adc834eb69/egress-router-binary-copy/0.log" Sep 05 03:10:00.773203 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:00.773181 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5q89w_75cd35e8-1c2c-4bc6-a121-15adc834eb69/cni-plugins/0.log" Sep 05 03:10:00.834872 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:00.834848 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5q89w_75cd35e8-1c2c-4bc6-a121-15adc834eb69/bond-cni-plugin/0.log" Sep 05 03:10:00.882135 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:00.882112 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5q89w_75cd35e8-1c2c-4bc6-a121-15adc834eb69/routeoverride-cni/0.log" Sep 05 03:10:00.937200 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:00.937172 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5q89w_75cd35e8-1c2c-4bc6-a121-15adc834eb69/whereabouts-cni-bincopy/0.log" Sep 05 03:10:00.964561 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:00.964537 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5q89w_75cd35e8-1c2c-4bc6-a121-15adc834eb69/whereabouts-cni/0.log" Sep 05 03:10:01.122117 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:01.122020 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r47jj_257406f0-bf02-4fea-a10b-daf9750ddefc/network-metrics-daemon/0.log" Sep 05 03:10:01.149742 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:01.149717 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r47jj_257406f0-bf02-4fea-a10b-daf9750ddefc/kube-rbac-proxy/0.log" Sep 05 03:10:02.131651 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:02.131623 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8cc78_306ce2c3-da47-4ecd-965a-5d2bda450872/ovn-controller/0.log" Sep 05 03:10:02.150466 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:02.150440 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8cc78_306ce2c3-da47-4ecd-965a-5d2bda450872/ovn-acl-logging/0.log" Sep 05 03:10:02.176287 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:02.176267 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8cc78_306ce2c3-da47-4ecd-965a-5d2bda450872/kube-rbac-proxy-node/0.log" Sep 05 03:10:02.199758 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:02.199731 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8cc78_306ce2c3-da47-4ecd-965a-5d2bda450872/kube-rbac-proxy-ovn-metrics/0.log" Sep 05 03:10:02.222723 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:02.222702 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8cc78_306ce2c3-da47-4ecd-965a-5d2bda450872/northd/0.log" Sep 05 03:10:02.242437 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:02.242419 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8cc78_306ce2c3-da47-4ecd-965a-5d2bda450872/nbdb/0.log" Sep 05 03:10:02.268134 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:02.268116 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8cc78_306ce2c3-da47-4ecd-965a-5d2bda450872/sbdb/0.log" Sep 05 03:10:02.308542 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:02.308485 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8cc78_306ce2c3-da47-4ecd-965a-5d2bda450872/ovnkube-controller/0.log" Sep 05 03:10:03.484090 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:03.484058 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-685456655f-7f8gm_00e747c8-fb32-431b-ab52-98b738cca93e/check-endpoints/0.log" Sep 05 03:10:03.534896 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:03.534873 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-jslqd_b5be9e27-b6e4-4477-9c67-2245faef62e9/network-check-target-container/0.log" Sep 05 03:10:04.491601 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:04.491576 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-rrpll_3132f740-a0ad-4ea5-be63-aaff35aa174f/iptables-alerter/0.log" Sep 05 03:10:05.223090 ip-10-0-128-63 kubenswrapper[2567]: I0905 03:10:05.223066 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-5fc7x_a4cb2d0a-0b28-4d06-a48f-ce729554a02e/tuned/0.log"